Operators & self-hosting

Hosted vs. your own instance

The operator dashboard (/operator) works the same on both. The rest of this page is for people running their own instance from the source repository.

Configuration

The API reads environment variables (or a .env file in the repository root). Anything other than ENVIRONMENT=dev|development|test|testing counts as production, and then the API refuses to start unless:

• SECRET_KEY is random and at least 32 characters (openssl rand -hex 32);
• CORS_ORIGINS lists explicit origins (JSON array or comma-separated; no *);
• PUBLIC_BASE_URL uses https://;
• PAYSTACK_SECRET_KEY, if set, is a test key, unless PAYSTACK_ALLOW_LIVE=true.

.env.example lists the minimum set. Billing intervals, payment limits and rate limits have sensible defaults; see server/app/core/config.py and docs/OPERATIONS.md for every setting.

Deploy

Either run the whole stack with Docker Compose:

git clone https://github.com/meshach5667/AgentMeshh.git && cd AgentMeshh
export SECRET_KEY=$(openssl rand -hex 32)
export PUBLIC_BASE_URL=https://api.example.com FRONTEND_URL=https://app.example.com
docker compose up -d

or deploy the parts separately: the API from Dockerfile.server (or uvicorn app.main:app in server/) on any host with PostgreSQL, and the dashboard from client/ on Vercel with NEXT_PUBLIC_API_URL set. Put the API behind TLS, run uvicorn with --proxy-headers, and restrict --forwarded-allow-ips to your proxy. Change the default database credentials in docker-compose.yml.

Database migrations

With PostgreSQL, the API runs alembic upgrade head on startup, holding a database lock so several replicas cannot migrate at once. If a migration fails, the API does not start. To run them yourself instead, set MIGRATE_ON_STARTUP=false and run:

cd server && alembic upgrade head

SQLite is for local development only: create it with python scripts/init_dev_db.py, since some migrations cannot run on SQLite.

First operator

Operators cannot self-register. On a machine with the same DATABASE_URL as the API (for example your host's shell):

cd server
python scripts/create_operator.py ops@example.com              # prompts for a password (12+ characters)
python scripts/create_operator.py lead@example.com --role admin

Running it for an existing account promotes that account. Each use is recorded in the audit log as user.operator_provisioned. Operator accounts have no multi-factor authentication yet; protect them carefully.

Model catalog

Under Model Catalog, approve models and enable or disable their artifacts (one per runtime). New catalog entries start unapproved. Adding an artifact, which needs a pinned source (an Ollama tag, or a Hugging Face commit and, for llama.cpp, a file SHA-256), is done with POST /models/{id}/artifacts; there is no dashboard form for it yet.

Pricing

Under Pricing, publish an immutable price version, effective now or later: CPU core-hour, memory GB-hour, GPU-memory GB-hour, input and output tokens per million, and the platform fee. Until a price list exists, usage is measured but not charged. You can set token prices to zero to charge for resources only.

Payments (Paystack)

1. In the Paystack dashboard (test mode), copy the secret key into PAYSTACK_SECRET_KEY and set the webhook URL to https://<your-api>/api/v1/payments/paystack/webhook.
2. Check the integration against Paystack's real test API: PAYSTACK_SECRET_KEY=sk_test_... python scripts/paystack_smoke.py (add --pay to pay with a test card).
3. Make a top-up from the dashboard and confirm that the webhook arrives: Reconciliation should show no open top-ups, and Ledger should show the deposit.
4. Payouts use Paystack Transfers: they need a funded balance and transfer OTP turned off, or they stop at OTP_REQUIRED.

Top-ups are credited once, only after Paystack reports success with a matching amount and currency. Payouts are held at request time and need operator approval (Payouts). Reconciliation runs every 10 minutes when payments are enabled.

Monitoring

Logs are JSON lines with a request id; API keys, node credentials, bearer tokens and Paystack keys are redacted. Alert on readiness failing, rising 5xx, payouts stuck PENDING or OTP_REQUIRED, reconciliation reporting ok: false, and Background job … failed in the logs.

Backups and recovery

cd server
DATABASE_URL=postgresql+asyncpg://… KEEP=14 ops/backup_db.sh /var/backups/agentmesh
DATABASE_URL=postgresql+asyncpg://…/agentmesh_restore ops/restore_db.sh /var/backups/agentmesh/agentmesh-<ts>.dump

• Backups are compressed pg_dump files (mode 0600). Store copies off the host, encrypted, and test a restore monthly. The restore script refuses a non-empty database.
• After a restore: point DATABASE_URL at it, start the API (migrations run), then run reconciliation to re-check open top-ups and payouts with Paystack.
• API restarts lose nothing: workers reconnect and billing resumes from each deployment’s watermark without double charging.
• Never refund a pending payout by hand; the transfer may still complete. Ledger corrections and refunds need a code path that does not exist yet.
• Full runbooks are in docs/OPERATIONS.md.

Publishing worker releases

1. Set WORKER_VERSION in server/node_agent/config.py, and WORKER_VERSION in client/src/lib/config.ts to match (a test checks this).
2. Commit, then tag and push: git tag worker-v0.3.0 && git push origin worker-v0.3.0.
3. The Worker release workflow checks the tag matches, builds the wheel, checks it installs, writes SHA256SUMS, and creates the GitHub release with the wheel, install.sh, uninstall.sh and SHA256SUMS.
4. Redeploy the dashboard so Add a machine points at the new version.

Signed self-updates are a separate, optional channel: sign a release manifest with scripts/sign_worker_release.py using an Ed25519 key kept offline, and point WORKER_RELEASE_DIR at the output. Never add that private key to GitHub Actions or the control plane.