Docker Compose
Run one Tenant with nylorun start and Docker Compose.
One Tenant, started with nylorun start. Compose runs the Runtime and its
stores. There are no replicas.
npx nylorun start --tenant appProvision
Install Node 24 and Docker Compose v2. A running Tenant uses about 1.2 GB, most of it Restate.
npx nylorun start --tenant app
npx nylorun statusstart opens no browser; it prints the Runtime and Studio URLs. Reach Studio
through an SSH tunnel to its loopback port. Studio uploads a message's
attachments through its own server, which sets no size cap of its own: each
upload is held to the Tenant's artifact limits (100 MiB per file by default).
Reverse proxy
nylorun start publishes the Runtime on 127.0.0.1 only. Put a TLS reverse
proxy on the same machine:
| Rule | Why |
|---|---|
Listen with TLS; forward to 127.0.0.1:<port> | The application key travels on every request. |
Rewrite Host to localhost:<port> | The Runtime answers 421 to any other Host. |
Forward only /health, /ready, /.well-known/oauth-protected-resource, and /v1/* | Nothing else is the Runtime or Management API. The well-known document tells OAuth clients where to sign in. |
| Never proxy Studio or Restate | They stay on the machine. |
| Do not buffer responses | Event streams stay open with a keepalive every 15 seconds. |
| Allow request bodies as large as the largest upload you allow | An artifact upload is up to the Tenant's artifact file limit (100 MiB by default). A skill or runtime file that nylorun apply or client.agentManifests.save() uploads is up to the Tenant's skill limit (2 GiB by default), and an agent manifest's files are up to its folderBytes limit (50 MiB by default) together. An AG-UI or A2A message carries files inline, about 28 MiB at the default limits. |
Pass Authorization and Nylorun-* through | Your app sets those headers. |
Pass 401 answers and WWW-Authenticate through unchanged | A refused key or token is a 401 with a Bearer challenge; clients need both. |
Start the Runtime with NYLORUN_PUBLIC_URL set to the proxy's URL | Webhook triggers' webhookUrl and the OAuth metadata name it: NYLORUN_PUBLIC_URL=https://runtime.example.com npx nylorun start. Without it they name http://localhost:<port>. |
Webhooks: publish only /v1/webhooks/* to the internet, and rate-limit it | A webhook trigger's deliveries arrive at POST /v1/webhooks/{endpointId} with no Authorization and no Nylorun-Protocol; a Standard Webhooks signature authenticates each one. Keep every other path private. |
runtime.example.com {
@api path /health /ready /.well-known/oauth-protected-resource /v1/*
handle @api {
reverse_proxy 127.0.0.1:8787 {
header_up Host localhost:8787
flush_interval -1
}
}
handle {
respond 404
}
}When a webhook sender is on the internet and the Runtime is not, a second site can publish the webhook path alone:
hooks.example.com {
handle /v1/webhooks/* {
reverse_proxy 127.0.0.1:8787 {
header_up Host localhost:8787
}
}
handle {
respond 404
}
}Give the sender https://hooks.example.com/v1/webhooks/<endpointId>. The
Runtime's webhookUrl names NYLORUN_PUBLIC_URL, so when webhooks have a host
of their own, keep only its path.
| Placement | Certificate |
|---|---|
| Same LAN | Tailscale, or a local CA (tls internal) whose root the app server trusts. Never plain HTTP. |
| Internet | A public certificate. Prefer a private path (VPN or the same network) over a public endpoint. |
Firewall so only the proxy port and your SSH port are reachable.
When the Runtime trusts your identity provider's tokens (an identity file), it
publishes those issuers at /.well-known/oauth-protected-resource; without an
identity file, that path is a 404. The document's resource is
NYLORUN_PUBLIC_URL. Set it to the proxy URL clients use, such as
https://runtime.example.com, when you run nylorun start; without it,
nylorun start sets http://localhost:<port>.
An identity file grants Agents with each issuer's agents: list and the
agents:write scope:
issuers:
- name: keycloak
issuer: https://sso.acme.dev/realms/eng
audience: https://runtime.example.com
jwks: https://sso.acme.dev/realms/eng/protocol/openid-connect/certs
allowedScopes: [agents:read, sessions:own, agents:write]
agents: ["users/{sub}/*", support]agentslists the Agents an issuer's tokens reach: exact ids, prefixes ending in/*, templates such as{org_id}/*, or*. An issuer withoutagentsreaches no Agent.- A file with an
environments:orsandboxes:key, or theenvironments:writeorsandboxes:writescope, stops the Runtime with a message naming the new spelling. - A token never saves agent manifests:
agent-manifests:writeis an application key's. - A token may be up to 16 KiB, the most a request's headers hold.
App server
Give the app server an application key of its own. On the Runtime's machine:
npx nylorun tenant keys put app-serverThe key prints once; keep it in the app server's secret store. On the app machine, set:
| Variable | Value |
|---|---|
NYLORUN_RUNTIME_URL | The proxy URL, https://runtime.example.com. On the Runtime's machine, the Runtime URL nylorun status prints. |
NYLORUN_SERVER_KEY | The application key tenant keys put printed. Never a management key or the admin key. |
In the project directory on the Runtime's machine, the SDK reads the project
link that nylorun start wrote and needs neither variable.
Save the agent manifests and make their Agents from a deploy step:
npm start in a generated project, or npx nylorun apply <dir> with the two
variables set. See Saving manifests. The app server
runs no code for the Runtime: the services your
HTTP tools call must be reachable from the Runtime's
machine. See Serve your users.
Private addresses
The gateway's requests to URLs that agents name (HTTP tools and remote MCP
servers), to a credential's via, and those of MCP server previews follow
these settings. Each is checked on the address actually connected to, and no
redirect is followed.
| Setting | Values | Default |
|---|---|---|
NYLORUN_ENDPOINT_PRIVATE | allow or refuse: private, loopback, and link-local addresses | allow |
NYLORUN_ENDPOINT_HTTP | allow or refuse: plain http URLs | allow |
NYLORUN_ENDPOINT_LOOPBACK | docker-host: localhost means the machine that runs Docker | Unset; a local Tenant sets it |
On a server, set NYLORUN_ENDPOINT_PRIVATE=refuse and
NYLORUN_ENDPOINT_HTTP=refuse on the gateway and the runtime, so a tool's URL
or a via cannot point the gateway at your internal network. If your tools' services live
on a private network, keep allow and limit the gateway's egress with your
firewall instead.
The workload token
A platform that runs Runtimes can give each one a short-lived token and rotate
it through a file, so its gateways trust the workload instead of a stored key.
A workload credential sends
the token to an MCP gateway, and a
custom model endpoint with
auth: { type: "workload" } sends it as the endpoint's bearer. Point the
Runtime at the file:
| Variable | Value |
|---|---|
NYLORUN_WORKLOAD_TOKEN_FILE | The token's file. The Runtime reads it at each use and keeps nothing, so a rotation applies to the next request with no restart. A missing, unreadable, or empty file fails the request with a message naming the variable; nothing is sent without the token. |
NYLORUN_WORKLOAD_TOKEN_ORIGINS | The comma-separated origins a workload credential may send the token to, such as https://connections.example.com: each https, or http only to a loopback host. Checked at each request: a credential whose via has another origin is refused at discovery. The model endpoint needs no entry, since its baseUrl is yours. |
Set both on the gateway container, which makes model and tool calls, and on
the runtime container of a Runtime that runs without the gateway. Mount the
file read-only.
nylorun start does not pass these variables to its containers yet, so a
local Tenant cannot use the workload token: set them in a Compose file or
deployment you run yourself.
The token stays in the process that makes the call. It never reaches a sandbox
(a sandbox's environment has no NYLORUN_* variable), the model, an event, or
a log, and it is scrubbed from what a server or the provider echoes, like any
credential value. A gateway that refuses it answers credential_rejected, and
the call's tool.completed names the gateway's origin in error.gateway.
Operate
| Service | Role |
|---|---|
runtime | Tenant API and scheduling. |
gateway | Model and tool calls, with the files a model call names. Holds provider keys. |
harness | Agent turns and workspaces. |
studio | Operator dashboard on loopback. |
postgres | Sessions, agent manifests, Agents, their memories, and their triggers. |
restate | Session advancement and trigger timers. |
s2-lite | Event history. |
rustfs | File bytes. |
npx nylorun status
npx nylorun local logs runtime -f
curl -sS http://127.0.0.1:8787/health
curl -sS http://127.0.0.1:8787/readyGET /health reports version, protocol range, and features. GET /ready
checks the listener, Tenant, Postgres, and Restate; it answers 503 until
they are ready. It does not check S2: an S2 outage affects only history and
event-stream reads, and shows as streams.reachable: false in GET /v1/tenant.
Triggers and wakes fire from the Runtime's durable
timers, with Postgres holding when each one is due; there is no extra process
to run. A stopped Tenant fires nothing. At the next start, each overdue trigger
fires once, marked late, and the times it missed are not run.
A 500 internal_error answer is logged as request_failed (keys_request_failed
for a key operation), with the method, the route's pattern, and the error's
message and stack; never a request body, header, or argument.
Configure the Tenant through the Management API (/v1/tenant/*) with a
management key and @nylorun/admin, or with npx nylorun on the machine.
Sizes, counts, and deadlines that requests run into are the Tenant's
Runtime limits, set the same way. Issue keys on the
machine:
npx nylorun tenant keys put backend # the app server's application key
npx nylorun tenant keys put ops --management # a management key, for @nylorun/adminA self-hosted Runtime container runs the same jobs with
nylorun-operate keys put <id> [--role management] and
nylorun-operate status.
Host tuning
NYLORUN_TUNING='{"tenant":{"upgradeStartMs":1200000},"pods":{"readyDeadlineMs":600000}}' \
npx nylorun start --tenant appThe Runtime's own timings, retries, poll intervals, and internal sizes are its
Host tuning: one JSON object in NYLORUN_TUNING, read once when a process
starts. Set only what you change; everything left out keeps its default.
nylorun startpassesNYLORUN_TUNINGfrom its shell to the runtime, gateway, and harness containers. Elsewhere, give every container of the deployment the same value: each reads the groups it uses. A pod's engine gets theharnessandenginegroups, as JSON of at most 4096 bytes.- An unknown key, a value of the wrong kind, or one past its ceiling stops the
process with the key's path, such as
NYLORUN_TUNING: tenant.openTimeoutMs: Too small: expected number to be >=1. - A value in milliseconds (
…Ms) is a whole number from 1 to 2,147,483,647. A backoff is{ "minMs", "maxMs" }: the first wait, doubled after each failure up to the longest.
| Group | What it tunes |
|---|---|
keys | Token lifetimes, clock tolerance, the egress certificates, and the keys service. |
harnessApi | The Harness API listener: joins, run offers, and segment rollover. |
api | The AG-UI heartbeat and the event-stream keepalive. |
host | Shutdown grace and readiness checks. |
objectStore | Multipart part size and retries. |
tenant | Opening the Tenant, leases, Retry-After, and moving pod Agents to new versions. |
sweep | The Tenant's sweep and outbox. |
triggers | Late fires and webhook timestamps. |
execution | Advance deadlines and wake de-duplication. |
restate | Sends, handler retries, and registration. |
postgres | Pool sizes and timeouts, the control bus, and replication. |
streams | S2 and the stream relay. |
gates | Gateway timeouts, MCP connections, and model-call reuse. |
egress | Egress gateway tunnels and timeouts. |
pods | Sandbox pod readiness, polling, and the memory floor. |
harness | Harness connections, MCP listing, sandbox helpers, and pod setup. |
traces | Trace export batching and retries. |
engine | Our engine's options, such as compaction and tool results. |
The settings an operator is most likely to change:
| Setting | Default | What it sets |
|---|---|---|
tenant.upgradeStartMs | 10 min | How long a pod may take to start on a new version before the Agent goes back to the one it ran. |
tenant.upgradeIdleMs | 1 min | How long a pod Agent waiting to start on a new version stays idle before it stops (its own lifecycle.idle when shorter). |
tenant.retryAfterSeconds | 5 | The Retry-After of a request refused while its Agent is busy or pods are not ready. |
triggers.webhookToleranceMs | 5 min | The clock skew a webhook delivery's webhook-timestamp may have, either way. |
triggers.lateGraceMs | 2 min | How long after it was due a trigger's fire counts as late. |
pods.readyDeadlineMs | 5 min | How long a pod may take to be ready before its start fails. |
pods.memoryFloorMiB | 128 | The least memory a sandbox gets; a smaller request rounds up. |
keys.tokenTtlMaxSeconds | 900 | The longest life of a token the Runtime signs, such as a capability link. |
postgres.poolSize | 10 | The Tenant pool's connections. |
gates.clientTimeoutMs | 630 s | How long a model or tool call's connection to the gateway may stay silent. |
traces.batchSpans, traces.batchDelayMs, traces.retries | 512, 2 s, 3 | Spans per export post, how long a span waits for its batch, and retries of a failed post. |
Every setting, with its default and ceiling, is listed in
DEPLOYMENT.md.
nylorun's own waits are NYLORUN_CLI_TUNING; see The CLI's own waits.
Traces
Each turn is one OpenTelemetry trace: a root span for the turn, named for the
agent manifest and carrying the version the turn ran
(nylorun.manifest.version), and a span for each model call and tool call. A
sandbox's network connections are not spans: each is an agent.egress event on
its Agent's events (see Egress records).
Export is off until an OTLP endpoint is set on the runtime container.
nylorun start passes these variables to it, and to the gateway, from the
shell it runs in. The same endpoint takes the processes' live spans and
metrics; Monitor lists them, with /metrics and
sampling:
OTEL_EXPORTER_OTLP_ENDPOINT=http://host.docker.internal:4318 \
OTEL_SERVICE_NAME=support-runtime \
npx nylorun start --tenant app| Variable | Meaning |
|---|---|
OTEL_EXPORTER_OTLP_ENDPOINT | The collector's OTLP/HTTP base URL. Spans go to <url>/v1/traces as JSON. |
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT | The full traces URL, instead of the base URL. |
OTEL_EXPORTER_OTLP_HEADERS | key=value pairs separated by commas (values percent-encoded), sent with every post. |
OTEL_SERVICE_NAME | service.name on the resource. Default nylorun-runtime (the gateway's nylorun-gateway). |
OTEL_RESOURCE_ATTRIBUTES | More resource attributes, as key=value pairs. |
OTEL_INSTRUMENTATION_GENAI_CAPTURE_CONTENT | The content spans carry: NO_CONTENT (the default) or SPAN_ONLY, every kind. See Monitor, Content. |
NYLORUN_TRACE_CONTENT, OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT | Deprecated: read for this release only when the switch above is unset. |
A collector on the same machine is host.docker.internal from the container.
By default a span carries references to the session log, never content; with
SPAN_ONLY, each text is cut at the Tenant's traceContentBytes
(Runtime limits, 8 KiB by default). A span never carries a
credential value, a person's id, an identity header, or an idempotency key.
- Off the hot path. The Runtime reads committed events from its own
replication slot,
nylorun_trace_export, created at start, and posts batches of up to 512 spans, or every 2 seconds (traces.batchSpansandtraces.batchDelayMsin Host tuning). No turn waits for it. Without an endpoint there is no slot. - A collector that is down loses only its spans. A
429, a5xx, or a lost connection is tried three times with backoff (traces.retries); then the batch is dropped, logged astrace_export_dropped, and counted onnylorun.telemetry.dropped(exporter="projector"), and the slot moves on, so Postgres keeps no WAL for the collector. - Backfill. Read the turns a collector missed from
GET /v1/sessions/{id}/turns/{turnId}/trace(session.trace(turnId)) and post them to the collector's/v1/traces. The span ids are the same each time a turn is read or exported. - Lag is not reported: watch
nylorun_trace_exportinpg_replication_slots.
Reading a turn's trace and passing traceparent in from your app are on
Traces.
Custom harnesses
A harness of your own runs only as a pod Agent's image: its agent manifest
names it in harness.id, with runtime.kind: "pod" and an image. It needs
sandbox pods; see
Kubernetes → Custom harnesses.
Backups
Back up the Postgres and RustFS volumes together with keys/vault-kek.
Postgres holds agent manifests (with the file list of each version saved from
files), Agents, memories with every version, triggers and their
runs, session queues, and webhook signing secrets. RustFS holds file bytes,
among them every file those versions were saved from. Agents' secrets
and webhook signing secrets are sealed with the vault key: a restore without
keys/vault-kek loses them. Make a new webhook secret with
npx nylorun agents triggers new-secret <id> <trigger>.
nylorun stop keeps volumes. nylorun local reset deletes this Tenant's data.
Upgrades
Upgrade nylorun and run start. The Runtime applies its database migrations
at start. start refuses to run an older Runtime than the Tenant last ran
(--allow-downgrade overrides).
The Runtime speaks protocol 16, 17, and 18, and the SDKs send 18: upgrade
nylorun first, which brings the Runtime and Studio, then the SDKs. An older
SDK keeps working against a newer Runtime; a newer SDK against an older Runtime
throws IncompatibleRuntimeError. From nylorun 0.17, migrations 0037 (model
call cost by token kind) and 0038 (agent manifest sources) run at the first
start, and delete nothing; from 0.15, 0028 to 0038. Read
Compatibility before you upgrade.
From protocol 14 or older, the upgrade deletes Environments
Migration 0032 deletes every Environment with its pod's state, its disk, its disk versions, and its secrets. It drops saved agents and environment manifests, and closes every open session; their history stays readable.
-
Before upgrading, with the old release, export what you need. Snapshot each disk, find the folder artifact and version the snapshot made, and download it as a zip. Secret values cannot be exported: list the names and have the values at hand.
npx nylorun environments snapshot <id> npx nylorun environments versions <id> npx nylorun artifacts zip <artifactId> --version <n> --output <id>.zip npx nylorun environments secrets list <id> npx nylorun environments manifests get <name>Keep each agent's v7 manifest JSON and each environment manifest's JSON for
nylorun migrate v8. -
Before the first start, rename the identity file's
environments:toagents:and theenvironments:writescope toagents:write. -
After the upgrade, on a cluster, delete the dropped Environments' pods and their join Secrets before you make pod Agents:
kubectl delete sandboxes.agents.x-k8s.io,secrets -l app.kubernetes.io/managed-by=nylorun-sandboxes -n nylorun-sbx-<tenant>. A pod Agent made with an old Environment's id would otherwise take over its pod and volume. If a tool policy rule names a tool asserver__tool, put the policy again withserverandtool. Write agent manifests (npx nylorun migrate v8 <agent.json> --runtime <environment-manifest.json>, thennpx nylorun apply <dir>), make the Agents, and write their secrets.
nylorun start runs the ghcr.io/nylorun/runtime and ghcr.io/nylorun/studio
images its release pins. NYLORUN_RUNTIME_IMAGE and NYLORUN_STUDIO_IMAGE
replace them, for example with a local build.
| Setting | Meaning |
|---|---|
NYLORUN_WORKER_VERSION | The Worker's version on the runtime container: 1 to 64 letters, digits, ., _, or -. Default: the Runtime's version. Set it on an image you build whose Worker code changed without a new Runtime version. |
The image's entry point is /app/runtime/server/dist/host/main.js. A script
that runs a file in the image directly uses that path.
Your own Postgres
A local Tenant starts Postgres with these settings already. A Postgres you run yourself needs them too:
| Setting | Value | Why |
|---|---|---|
wal_level | logical | The Runtime reads committed events from replication slots. |
max_replication_slots, max_wal_senders | At least 2 (the default 10 is enough) | Two slots: nylorun_stream_relay, and nylorun_trace_export when traces are exported. |
max_slot_wal_keep_size | A few GB (nylorun start uses 4GB) | A stuck slot cannot fill the disk. |
| The Runtime's role | REPLICATION, plus read access to nylorun_streams | The relay reads the Tenant's events. |
On a managed Postgres, turn on its logical replication option, for example
rds.logical_replication on RDS.
Connect the Runtime to Postgres directly or through a pooler in session mode.
Cancels and session resets reach every Runtime process through LISTEN and
NOTIFY, which a pooler in transaction mode (PgBouncer's default) does not
deliver; a cancel then reaches another process only through a 5-second poll.