AI deployment agent

How it works

Customers describe a deployment in plain language (/customer/ai) and authorize limits. A Gemma 4 model, served by Ollama on an internal inference machine, turns the request into a structured plan and diagnoses failures. Everything with consequences runs in the control plane's own code: catalog checks, limits, prices, wallet balance, node selection, capacity reservation, submission, verification, and stopping.

The adapter connects outbound over HTTPS, reports health, and leases queued model calls from the control plane. The inference machine needs no public address and no open port, and Ollama stays bound to 127.0.0.1. The inference machine is never a provider node: it is not enrolled as a worker and never runs customer workloads.

Check the hardware

Nothing is assumed about the machine. On the inference machine, run:

lscpu | grep -E "Model name|^CPU\(s\)|Flags" | cut -c1-120   # CPU and AVX2/AVX-512 support
free -h                                                     # total and available memory
df -h /usr/share/ollama 2>/dev/null || df -h ~                # disk for models
lspci | grep -iE "vga|3d"                                     # GPUs (an Intel iGPU is not used by default)
cd server && python -m ai_adapter doctor                      # all of the above, plus Ollama and a recommendation

doctor changes nothing. It reports CPU flags, memory, free disk where Ollama stores models, whether an NVIDIA or ROCm GPU device exists (installed tools alone do not count), whether Ollama answers, and whether the configured model is pulled.

Reference machine (measured 2026-10-21):

Choose a Gemma 4 variant

Download sizes below come from the Ollama registry manifests (checked 2026-10-21). Memory figures are estimates (weights plus about 1.5 GB for a 4,096-token context and the runtime).

Larger variants (12b, 26b, 31b) are not supported by the adapter on laptops. Set the same tag in OLLAMA_MODEL on the control plane and the adapter: the control plane rejects answers from any other model, and the adapter refuses jobs for another model, so there is never a silent fallback.

Install Ollama and the model

1. Install Ollama (Linux): curl -fsSL https://ollama.com/install.sh | sh. This creates the ollama systemd service.
2. Optional but recommended: apply server/packaging/ollama-agentmesh.conf as a drop-in. It pins Ollama to loopback, keeps one model loaded with one request at a time, and caps it at 6 of 8 threads and 8 GB of RAM.
3. Set up the adapter environment (a Python 3.11+ venv in server/ with pip install -r requirements-worker.txt).
4. Pull the model through the adapter, which checks disk space and RAM first: python -m ai_adapter pull shows what it would do; python -m ai_adapter pull --yes downloads.
5. Test real inference: python -m ai_adapter check runs one small structured (JSON-schema) generation and prints the latency. It fails if the installed Ollama cannot run the model.

sudo mkdir -p /etc/systemd/system/ollama.service.d
sudo cp server/packaging/ollama-agentmesh.conf /etc/systemd/system/ollama.service.d/agentmesh.conf
sudo systemctl daemon-reload && sudo systemctl restart ollama

Connect it to AgentMesh

1. Generate a token: openssl rand -hex 32.
2. Control plane (e.g. Render environment): set AI_INFERENCE_TOKEN to it and OLLAMA_MODEL to the tag. Without a token the adapter endpoints are disabled and customers see “not configured”.
3. Inference machine: copy server/ai_adapter/ai-adapter.env.example to ~/.config/agentmesh/ai-adapter.env, fill in the token, and chmod 600 it.

Control-plane limits (server/app/core/config.py): AI_QUEUE_MAX_LENGTH (20), AI_QUEUE_TTL_SECONDS (3600), AI_MAX_ATTEMPTS (2), AI_RATE_LIMIT_PER_MINUTE (3 per user), AI_MAX_AGENT_STEPS (6), AI_MAX_TOOL_CALLS (8), AI_MAX_DEPLOYMENT_HOURS (168), AI_MAX_RECOVERY_ATTEMPTS (3).

Run it at startup

mkdir -p ~/.config/systemd/user
cp server/packaging/agentmesh-ai-adapter.service ~/.config/systemd/user/
# edit WorkingDirectory/ExecStart if the repository is not at ~/AgentMesh
systemctl --user daemon-reload
systemctl --user enable --now agentmesh-ai-adapter
loginctl enable-linger "$USER"   # keep running while logged out
journalctl --user -u agentmesh-ai-adapter -f

Monitor CPU and memory

• Every heartbeat reports CPU %, load per CPU, and available memory; operators see them in GET /api/v1/ai/health. Customers only see available / busy / offline and the queue length.
• On the machine: ollama ps (loaded model and its memory), systemctl status ollama, top or htop.
• When memory or CPU is short, the adapter stops leasing work and reports why; requests wait in the queue. It never restarts Ollama.
• Before the model is loaded, the adapter requires room for the weights plus 1.5 GB; once loaded (kept for 30 minutes), only the configured reserve.

Restarts and disconnections

The inference machine is a personal computer; it will be off or offline sometimes. When it is:

• After AI_NODE_TIMEOUT_SECONDS (45 s) without a heartbeat the AI service shows as offline. Nothing is sent to it.
• The control plane, database, billing, and every running customer deployment keep working. Deployment state lives in the database, not in the model conversation.
• New AI requests are accepted and queued (up to AI_QUEUE_MAX_LENGTH). A request not answered within AI_QUEUE_TTL_SECONDS fails with “No AI decision was made”; nothing is deployed.
• Deterministic steps continue: verification, duration and spend limits, stopping and cancellation all work without the model.
• When the machine reconnects, the adapter heartbeats, leases the queued requests, and runs continue where they stopped. A model call that was in progress is re-queued once its lease expires (bounded by AI_MAX_ATTEMPTS).

Autonomy limits

Customers set these per request; they are enforced in code, never by the prompt:

• Maximum spend: the plan’s estimate must fit, the wallet must cover it, and the deployment is stopped when rated charges plus unrated READY time reach it.
• Maximum duration: the deployment is stopped when it is reached.
• Maximum CPU cores, memory, and GPU memory (0 = no GPU).
• Maximum automatic recovery attempts (platform ceiling 3).
• Allowed runtimes (Ollama, llama.cpp, vLLM) and model sources (Ollama registry, Hugging Face).
• Agent steps and tool calls per run (platform settings).

Tests

cd server
pytest tests/test_ai_agent.py tests/test_ai_adapter.py     # scripted model answers, no Ollama needed

# Optional: real structured generation against a local Ollama that already has the model
AGENTMESH_TEST_OLLAMA_URL=http://127.0.0.1:11434 \
AGENTMESH_TEST_OLLAMA_MODEL=gemma4:e2b-it-qat \
  pytest tests/test_ai_adapter.py -k real_ollama

The tests never pull models, never create paid deployments (no price is configured unless a test sets one), and never use GPUs. The real-Ollama test is skipped unless both variables are set and the model is already present.

Not yet verified

• Gemma 4 has not been pulled or run on the reference HP 15, so its latency, memory use, and the quality of its JSON plans there are unmeasured. Run python -m ai_adapter check after pulling.
• Ollama 0.40.6 support for the gemma4 tags is assumed from the registry listing; check confirms it.
• The memory figures for each variant are estimates.
• The end-to-end flow was tested with scripted model answers and a simulated worker tunnel, not with a real provider node running a model.