Configure models and API keys
Your coding agent, Loom’s internal LLM and Loom’s embedding model are separate choices. You can connect Claude Code to a Loom server using OpenAI models, or Codex to a server using Claude. The agent does not need to use the same provider as Loom.
| Choice | What it does | Where to configure it |
|---|---|---|
| Your agent: Claude Code, Codex or another client | Runs your coding or assistant workflow and calls Loom’s tools. | In that client; connect it with a Loom token. |
| Loom’s LLM | Extracts facts, summarizes sessions and generates answers inside Loom. | The server’s provider, model and API key settings. |
| Loom’s embedding model | Makes memories, documents and code searchable by meaning. | The server’s embedding model, dimensions and provider key. |
LOOM_API_KEY is a token for accessing Loom. It is different from
OPENAI_API_KEY, ANTHROPIC_API_KEY and GEMINI_API_KEY, which pay for the server’s
model calls. Keep provider keys on the server; connecting an agent does not transfer its
subscription credentials or API keys to Loom.
What works without provider keys?
The released Docker installation embeds with local (bge-small, 384 dimensions, on CPU
from files the image carries) and selects the fake-echo-1 LLM, unless the installer
set a local model server (a model server on your
machine). It boots without provider keys or
network: storage, the dashboard, structural code indexing and semantic search work,
though a hosted embedder retrieves better on paraphrase-heavy questions. The fake LLM
does not produce meaningful extracted facts, summaries or generated answers. A
checkout’s .env selects fake-ngram-3072 embeddings, which match on character
overlap. An upgraded installation keeps the embedder its ~/.loom/.env names; switching
it to local is a model change like any other, and re-embeds what is stored.
For stronger retrieval over paraphrase-heavy content, configure a hosted embedding provider. For automatic fact extraction, summaries and generated answers, configure an LLM too. An LLM alone does not replace an embedding model; an embedding model alone does not generate answers. With no embedding configured, a checkout’s server uses its fake embedder; it does not skip embedding.
Some hosted providers also fall back to fakes when their key is missing. Check
fake_embedding, fake_llm, providers_degraded and problems in /v1/health, then
save and retrieve a test fact and try an answer. An invalid key or inaccessible model can
still cause provider calls to fail; a running server is not proof of working model access.
Use LOOM_REQUIRE_REAL_PROVIDERS=true in environment-based deployments to make fake or
keyless providers a degraded-health condition.
Choose a setup
| What your organization has | Loom LLM | Embeddings |
|---|---|---|
| OpenAI API access | An available OpenAI model | OpenAI; one API key can serve both roles. |
| Anthropic plus an approved embedding provider | An available Claude model | OpenAI, Gemini or a compatible endpoint, with its key. |
| Anthropic API access only | An available Claude model | Add an embedding provider and its API key. |
| AWS role with Bedrock model access | A permitted Bedrock model or inference profile | Cohere Embed v4 through Bedrock, or another configured embedder. |
| No provider key; a model server on the machine | A local model (local) |
The local embedder the image carries. |
One OpenAI API key is enough for both roles; an Anthropic key is not required. In Models & keys, choose one main LLM and leave the displayed per-job overrides blank to use it for those jobs, including capture verification and retrieval judging. Background settings such as the conflict resolver are configured separately; use the per-job recipe to route them explicitly.
Choose OpenAI, Google Gemini, Bedrock or an
OpenAI-compatible embedding endpoint. OpenAI embeddings use
an embedding model such as text-embedding-3-large, not a GPT chat model.
Claude can handle Loom’s LLM jobs, but requires a separate embedding provider and key.
Anthropic does not provide its own embedding model; Voyage is not a built-in Loom provider.
Anthropic’s embedding guidance
Loom’s internal model jobs can authenticate with provider API keys, gateway workload tokens or deployment AWS credentials. Local models can run without provider credentials. Coding-agent CLI logins and subscriptions do not configure Loom’s internal providers; your external agent’s choice of provider is independent.
Set the server configuration
Recommended: use the dashboard
As a platform administrator, open Settings → Models & keys (also available during Setup).
Choose your LLM and embedding provider, enter their model IDs and API keys, then click
Test & apply. Blank per-job overrides apply the main LLM to the displayed jobs;
the conflict resolver keeps its separate setting.
Saved dashboard settings override the environment and are encrypted in ClickHouse.
No Docker restart or Compose command is needed. Key and LLM updates apply live;
embedding changes start the rebuild described below. Use environment
configuration removes those overrides and restores this server’s startup settings;
it does not reload an edited .env file. Changing the embedding space requires a rebuild.
The dashboard’s embedding choices are hosted providers. To keep the released stack’s
local embedder and add a real LLM, set only the LLM lines in ~/.loom/.env
(Claude for the LLM shows them) and recreate the container: the
embedding space is unchanged, so nothing is re-embedded.
For dashboard embedding changes, Loom queues that rebuild automatically and keeps
data-plane requests paused until verification, publication and cleanup finish.
Models & keys shows the phase and progress; a failed job stays paused and offers
retry from its saved cursor. Keep the ClickHouse volume and the original
LOOM_JWT_SECRET, which seals the stored provider keys, when recreating containers. Rebuilds make provider calls and may
take time and incur fees proportional to the indexed content. The rebuild publishes
to local MergeTree tables only. The first embedding choice on an install with no
stored vectors, such as a fresh install that has stored nothing yet, switches
directly with no rebuild. Once vectors exist on replicated or shared storage, such
as ClickHouse Cloud, Models & keys locks
the embedding fields and the server refuses an embedding change before pausing
anything: change the embedding model in the environment instead and re-embed each
index as described above. Lost remote-process recovery remains a documented limitation in
provider reconfiguration.
The token settings are separate: they configure agent access to Loom, not model-provider credentials.
Alternative: configure the environment
For automated deployments, edit ~/.loom/.env (or the directory selected with
LOOM_INSTALL_DIR). For a source checkout, edit the repository’s .env.
Replace existing entries instead of appending duplicate keys. These settings are the
startup baseline; saved dashboard overrides take precedence. Environment edits require
container recreation or a native server restart, as described below.
OpenAI for both roles
LOOM_LLM_PROVIDER=openai
LOOM_LLM_MODEL=gpt-6-luna
OPENAI_API_KEY=<your OpenAI API key>
LOOM_EMBEDDING_PROVIDER=openai
LOOM_EMBEDDING_MODEL=text-embedding-3-large
LOOM_EMBEDDING_DIM=3072
LOOM_REQUIRE_REAL_PROVIDERS=true
The examples use Loom’s shipped model defaults. Choose a model enabled for your API account. Setting a key alone does not change the stock installer configuration, which names its providers explicitly (the local embedder and the fake LLM).
Claude for the LLM
LOOM_LLM_PROVIDER=anthropic
LOOM_LLM_MODEL=claude-haiku-4-5
ANTHROPIC_API_KEY=<your Anthropic API key>
LOOM_REQUIRE_REAL_PROVIDERS=true
Then choose an embedding configuration:
# OpenAI embeddings
LOOM_EMBEDDING_PROVIDER=openai
LOOM_EMBEDDING_MODEL=text-embedding-3-large
LOOM_EMBEDDING_DIM=3072
OPENAI_API_KEY=<your OpenAI API key>
# Or Gemini embeddings
LOOM_EMBEDDING_PROVIDER=gemini
LOOM_EMBEDDING_MODEL=gemini-embedding-001
LOOM_EMBEDDING_DIM=3072
GEMINI_API_KEY=<your Gemini API key>
For another service that implements the OpenAI embeddings API, configure its API root, model ID and native output width:
LOOM_EMBEDDING_PROVIDER=compatible
LOOM_EMBEDDING_BASE_URL=https://embedding-provider.example/v1
LOOM_EMBEDDING_MODEL=your-embedding-model
LOOM_EMBEDDING_DIM=768
LOOM_EMBEDDING_API_KEY=<your embedding provider API key>
# Enable only if the endpoint accepts the optional dimensions parameter:
LOOM_EMBEDDING_SEND_DIMENSIONS=false
The base URL is the API root; Loom appends /embeddings. Use HTTPS for remote
services. HTTP is supported for private self-hosted endpoints, but transmits the
key and content unencrypted. Compatibility requires indexed float-array responses
with the configured width; a provider using a different API needs its own adapter.
Changing the endpoint, model or width requires rebuilding existing embeddings.
If your organization currently has only Anthropic API access, obtain access to an embedding provider before enabling semantic search.
Amazon Bedrock
Native Amazon Bedrock uses the server’s AWS SDK credential chain, including deployment
roles, and needs permission to invoke the selected models or inference profiles. The
server image includes the AWS SDK; a source install needs its aws extra. Activate the
provider only after every server and worker supports it. AWS credentials stay in the
deployment; they are not entered as model-provider API keys in Models & keys.
LOOM_LLM_PROVIDER=bedrock
LOOM_LLM_MODEL=<model-or-inference-profile-id>
LOOM_BEDROCK_REGION=us-east-1
LOOM_REQUIRE_REAL_PROVIDERS=true
Replace the model ID and region with ones your deployment can invoke; there is no implicit Bedrock model. Keep the current embedder, or configure Cohere Embed v4 separately, using an available model or inference profile, for example:
LOOM_EMBEDDING_PROVIDER=bedrock
LOOM_EMBEDDING_MODEL=us.cohere.embed-v4:0
LOOM_EMBEDDING_DIM=1536
Changing the embedding provider requires rebuilding existing vectors. Missing or expired AWS credentials fail rather than selecting a fake provider. Native Bedrock cannot be used when an inference gateway is required. See native Bedrock for IAM resources, regional availability, supported dimensions and request limits, and per-job settings for model overrides.
Keep every LLM job on your chosen provider
Loom supports different models for different jobs, but you do not need accounts with
multiple LLM providers. Add these settings after LOOM_LLM_MODEL to have the
normally provider-specific jobs use your selected model:
LOOM_MEMORY_EXTRACT_VERIFY_MODELS=${LOOM_LLM_MODEL}
LOOM_CONFLICT_RESOLVER_MODEL=${LOOM_LLM_MODEL}
LOOM_SESSION_BRIEF_LLM_MODEL=
LOOM_CODE_BISECT_LLM_RERANK_MODEL=
LOOM_TRACKER_DUPCHECK_RERANK_MODEL=
LOOM_RETRIEVAL_JUDGE_MODEL=
Compose and Loom’s dotenv loader expand the verifier’s model from the earlier entry. The empty values above inherit the main LLM. This keeps the capture verifier enabled; an empty verifier list would disable it. Remove any other cross-provider model overrides you added previously, including fallback, synthesis and reranking models.
The conflict resolver can retire superseded memories. Validate a replacement judge
against your own conflicts before automatic resolution; set
LOOM_MEMORY_CONFLICT_RESOLVER_ENABLED=false while evaluating it.
These settings matter: some shipped advanced defaults name Claude or OpenAI models explicitly, and an override with no provider key can fall back to the fake client. Using one model across jobs is a convenience configuration, not the configuration behind every published benchmark. Tune individual jobs later using models per job.
A model server on your machine, no API key
LOOM_LLM_PROVIDER=local runs Loom’s LLM jobs on a chat-completions server you run:
Ollama, llama.cpp’s llama-server, or any server that speaks the
OpenAI chat-completions format. Loom sends it no key. With the released stack’s local
embedder, nothing leaves the machine.
A new install run from a terminal offers this when Ollama answers on localhost:11434;
LOOM_LOCAL_MODELS=ollama in front of sh loom-install.sh sets it without asking, on a
new install or an existing one, and LOOM_LOCAL_MODELS=off never offers it. It writes:
LOOM_LLM_PROVIDER=local
LOOM_LLM_MODEL=qwen2.5:3b
LOOM_LOCAL_LLM_BASE_URL=http://host.docker.internal:11434/v1
LOOM_ASSISTANT_PROVIDER=local
LOOM_ASSISTANT_MODEL=qwen2.5:3b
LOOM_ASSISTANT_ANSWER_CEILING_USD=0
LOOM_MEMORY_EXTRACT_VERIFY_MODELS=
LOOM_MEMORY_CONFLICT_RESOLVER_ENABLED=false
LOOM_LLM_REQUEST_TIMEOUT_S=900
LOOM_LLM_RETRY_ATTEMPTS=1
LOOM_EXTRACTION_QUEUE_MAX_BATCHES_PER_PASS=1
LOOM_EXTRACTION_QUEUE_MAX_PASS_SECONDS=3600
LOOM_EXTRACTION_QUEUE_MAX_ATTEMPTS=2
LOOM_ENTITY_EXTRACT_ENABLED=false
LOOM_SESSION_BRIEF_LLM_MODEL=
LOOM_CODE_BISECT_LLM_RERANK_MODEL=
LOOM_TRACKER_DUPCHECK_RERANK_MODEL=
LOOM_RETRIEVAL_JUDGE_MODEL=
The preset turns off the two jobs that remove facts on a model’s judgment, whose
thresholds were set on hosted models: the capture verifier, which drops a fact every
verifier model calls not durable, and the background
conflict resolver,
which retires an older row it judges replaced. A small model’s misjudgment there would
lose facts. Writes still flag conflicts for the Review queue; set
LOOM_MEMORY_CONFLICT_RESOLVER_ENABLED=true with the key for
LOOM_CONFLICT_RESOLVER_MODEL (gpt-6.1-sol, OpenAI) to judge them. On an existing
install, a configuration saved in Models & keys wins over .env: after the restart,
choose Use environment configuration there for the preset to apply.
Pull the model first (ollama pull qwen2.5:3b, about 2 GB), and give Ollama a context
window that holds Loom’s prompts: extraction sends about 15,000 tokens, and Ollama cuts a
longer prompt to its window without an error, so run it with
OLLAMA_CONTEXT_LENGTH=32768 (the window qwen2.5 takes). A CPU can take minutes to
read a prompt that long, so the preset waits LOOM_LLM_REQUEST_TIMEOUT_S=900 seconds for
a reply rather than the hosted default of 300, and tries it once
(LOOM_LLM_RETRY_ATTEMPTS=1): a server that timed out answers no faster a second time.
The worker that replays the captures the coding plugins send gives a pass a deadline, so
the preset lets a deferred-extraction pass run an hour on one batch
(LOOM_EXTRACTION_QUEUE_MAX_PASS_SECONDS, by default four of its intervals, at least 60 s
and at most 900; a batch’s lease is kept at least a minute longer). On a local provider
a capture cut short (at that deadline, or by Loom shutting down) or whose call ran out its
timeout spends one of its LOOM_EXTRACTION_QUEUE_MAX_ATTEMPTS=2 attempts, so a session
too long for the machine ends failed (memory.extraction_status shows cancelled or
the timeout) instead of holding the queue: a long session can need several calls in a
row, each up to the timeout. On a server that answers one request at a time the timeout
also counts the wait behind other calls, and the assistant and memory.ask share the
server, so heavy use of them during a capture can fail it too; raise
LOOM_EXTRACTION_QUEUE_MAX_ATTEMPTS if it does. Entity tagging
(LOOM_ENTITY_EXTRACT_ENABLED) is off: it would queue behind extraction on a server that
answers one request at a time, and its pass, cut at its deadline, writes nothing, so
search does without entity matches. The other workers that
call a model are not tuned for a local one: observe extraction (off by default), whose
pass of up to 10 sessions is cut at 900 s and starts again from the newest session, and
conversation titles and structured profiles, which run once conversation logging or a
profile schema is set up.
For llama.cpp, point LOOM_LOCAL_LLM_BASE_URL at its /v1 root
(http://host.docker.internal:8080/v1) and set LOOM_LLM_MODEL to the name it serves;
for a local assistant, also set LIBRECHAT_LOCAL_LLM_BASE_URL to that root and
LOOM_ASSISTANT_MODEL to that name. A server run from a checkout reaches the model at
localhost, the setting’s default (http://localhost:11434/v1).
A per-job model whose name starts with gpt, o1, o3, o4, chatgpt or claude goes
to that hosted provider whatever LOOM_LLM_PROVIDER says; a local model named that way
(gpt-oss) runs locally as LOOM_LLM_MODEL, and in a per-job setting only when that
setting names the same model.
- Reaching the host. The containers reach the machine as
host.docker.internal. Docker Desktop forwards that to services on loopback; with Docker Engine on Linux, Ollama must listen beyond loopback (OLLAMA_HOST=0.0.0.0, with port 11434 closed to the network). Under the Helm chart withnetworkPolicy.enabled, add the server’s port tonetworkPolicy.extraEgress. - The assistant.
LOOM_ASSISTANT_PROVIDER=localmoves the assistant’s agent to LibreChat’slocalendpoint, which callsLIBRECHAT_LOCAL_LLM_BASE_URL(defaulthttp://host.docker.internal:11434/v1), and needsLOOM_ASSISTANT_MODEL. The assistant answers by calling Loom’s tools, so pick a model the server runs with tool calls (Ollama marks them). A local model has no price, so each answer countsLOOM_ASSISTANT_ANSWER_CEILING_USD($0.10 by default) against the assistant’s share of the namespace’s daily model cost cap, as any unpriced model does: at the default cap and share, five answers a day. The preset sets the ceiling to0, so local answers cost nothing; it is also what an answer cut before its usage arrived costs, so keep it at its default when a hosted model serves the assistant. Under the Helm chart, addLOCAL_LLM_BASE_URLto the assistant’s Secret; the chart’s policy lets LibreChat reach only DNS, its database, Loom and TCP 443, so the model server needs a NetworkPolicy of your own that selects the LibreChat pod. - Quality and speed. Loom’s prompts and thresholds are tuned on hosted models, and no
published benchmark covers a local one, so treat a small model as a way to try Loom
with no key or to keep everything on the machine, not as an equal: expect extraction,
memory.askanswers and the assistant’s tool use to be weaker, by an amount nobody has measured. A small model on a CPU is slow on long prompts such as extraction, so a busy capture queue backs up; a GPU, or a larger model on a machine with the memory for it, helps. For a team server, a hosted model answers better.
Through an inference gateway
An OpenAI-compatible proxy or inference gateway can serve Loom’s chat-completions requests. Select the provider and a model the gateway serves, as well as its API root and key; a key alone does not change an explicitly configured fake provider:
LOOM_LLM_PROVIDER=openai
LOOM_LLM_MODEL=<gateway-model-id>
LOOM_OPENAI_BASE_URL=https://gateway.example/v1 # chat and OpenAI embeddings
LOOM_ANTHROPIC_BASE_URL=https://gateway.example # Messages: <root>/v1/messages
OPENAI_API_KEY=<the key the gateway issued for Loom>
For an Anthropic-format gateway, select LOOM_LLM_PROVIDER=anthropic, its model ID
and ANTHROPIC_API_KEY instead. Configure embeddings separately using a supported
provider and its model; a chat endpoint does not imply embedding support. Check
per-job overrides too. Both API roots
above must use an allowed host when the gateway is required, even for an unused provider.
Off its default, the Anthropic key is also sent as Authorization: Bearer, the
scheme gateways take. With a gateway key, require the gateway, so a wrong root can
never send prompts and the key anywhere else. A token exchange requires it by itself.
LOOM_INFERENCE_GATEWAY_REQUIRED=true
LOOM_INFERENCE_GATEWAY_HOSTS=gateway.example # comma-separated; exact names, any case
While the gateway is required, model calls and the memory content Loom posts
to classifiers and rerankers go only to the hosts LOOM_INFERENCE_GATEWAY_HOSTS
lists. A host matches exactly: no wildcard, and gateway.example does not
admit api.gateway.example or gateway.example.attacker.test; an IP address
matches only if it is listed.
- At start, Loom refuses to load with no
LOOM_INFERENCE_GATEWAY_HOSTS, or whileLOOM_OPENAI_BASE_URL,LOOM_ANTHROPIC_BASE_URL,LOOM_EMBEDDING_BASE_URL,LOOM_INFERENCE_TOKEN_URL(andLOOM_LOCAL_LLM_BASE_URLwhileLOOM_LLM_PROVIDER=local) or an entry ofLOOM_RERANK_ENDPOINTS_JSONnames another host, while the embedder is Gemini’s, or whileLOOM_CODE_SEARCH_RERANK_DEFAULTorLOOM_KNOWLEDGE_SEARCH_RERANK_DEFAULTisbuiltin:cohere. Both API roots count, the one for a provider you do not use included (their defaults are the providers’ own APIs): a model named per request may be either provider’s. - At call time, a model call to any other host is refused before it
leaves, on every redirect hop too, whatever named it: a model named per
request, a
LOOM_CODE_SEARCH_VECTOR_MODELSentry. A refused call is not retried, and/v1/healthreports the provider as failing. - Namespaces:
namespace.registerrefuses aclassify_endpointthat is not https at the default port of a listed host (422), and one stored before is never called: memory.set keeps the caller’s type, with a warning in the log.builtin:cohereis refused as a rerank choice (422), and a namespace that stored it reranks with nothing, with a warning in the log. - Dashboard: saving a model configuration with an endpoint at another host
(or the Gemini embedder) answers 422, and the model check answers
gateway_required. A stored one that the check refuses is not applied, its serving vector settings included, and/v1/healthreportsmodel_configuration_refused. The check reads only the embedding, so a refused configuration always embeds differently from the settings in force, and the vector tables may hold that other space: every data-plane request on that server is refused (503GATEWAY_REQUIRED, which neither SDK retries, with noRetry-After) and the server is not ready, until a configuration the check accepts replaces it or it is reset to the environment’s. A rolling restart then stops at that server: replace the configuration before rolling. - Rolling it out: a previous build ignores both settings, so turn them on
once every replica runs this one. A deployment that already exchanges a token
refuses to start on this build until
LOOM_INFERENCE_GATEWAY_HOSTSlists its gateway, and while it embeds with Gemini, names a rerank endpoint at another host or defaults code search tobuiltin:cohere; requests namingbuiltin:cohereget a 422, and namespaces that stored it or an off-gatewayclassify_endpointstop reranking or classifying until it is changed. Set the hosts and move the embedder behind the gateway (LOOM_EMBEDDING_PROVIDER=compatible,LOOM_EMBEDDING_BASE_URLat the gateway) first, or the rollout stops at the first upgraded replica.
A gateway that accepts workload identity (RFC 8693 token exchange) needs no key at all: Loom exchanges its workload’s OIDC token there, renews the answer five minutes before it runs out (halfway, for one living under ten minutes), and sends it with every call. With no key to infer them from, name the provider and the models:
LOOM_INFERENCE_TOKEN_URL=https://gateway.example/oauth/token # https
LOOM_INFERENCE_GATEWAY_HOSTS=gateway.example # every root's host
LOOM_INFERENCE_WORKLOAD_TOKEN_FILE=/var/run/secrets/tokens/gateway # rotated by the platform
LOOM_INFERENCE_TOKEN_SCOPE="chat embeddings" # if the gateway asks for one
LOOM_OPENAI_BASE_URL=https://gateway.example/v1 # both roots required:
LOOM_ANTHROPIC_BASE_URL=https://gateway.example # the token is the gateway's
LOOM_LLM_PROVIDER=openai
LOOM_LLM_MODEL=gpt-6-luna
LOOM_EMBEDDING_MODEL=text-embedding-3-large
Loom refuses to start while either provider’s root, or the token endpoint,
is at a host LOOM_INFERENCE_GATEWAY_HOSTS does not list: a model named per
request may be either provider’s, and the gateway’s token would go there. In
the dashboard, model changes are checked through the gateway too.
On Kubernetes the token file is a projected service-account token with the
gateway’s audience; the gateway must trust the cluster’s OIDC issuer and match
Loom’s service account (system:serviceaccount:<namespace>:<name>), not just
the issuer, since every pod of the cluster can get a token from it.
A gateway that verifies the workload’s token itself takes it as the bearer, with no exchange: set the bearer mode instead of a token endpoint. Loom reads the file at most once a minute, and again at once when the gateway answers 401, so a rotated token is picked up without a restart:
LOOM_INFERENCE_WORKLOAD_TOKEN_MODE=bearer
LOOM_INFERENCE_WORKLOAD_TOKEN_FILE=/var/run/secrets/tokens/gateway # sent as it is
LOOM_INFERENCE_GATEWAY_HOSTS=gateway.example # required, as with the exchange
LOOM_OPENAI_BASE_URL=https://gateway.example/v1
LOOM_ANTHROPIC_BASE_URL=https://gateway.example
LOOM_EMBEDDING_PROVIDER=compatible # embeddings through the gateway too
LOOM_EMBEDDING_BASE_URL=https://gateway.example/v1
Name the provider and the models as in the exchange example. The token goes only to the
gateway’s hosts: the bearer mode requires the gateway, as the exchange does. A compatible
embedding endpoint given its own LOOM_EMBEDDING_API_KEY sends that key instead.
Naming the customer on every call
A gateway that meters or budgets per customer reads the customer from a header. Name the headers, and every chat, embedding, rerank and classify request carries the org whose data it holds, and the feature that made it:
LOOM_INFERENCE_CUSTOMER_HEADER=X-Customer-Id # the org
LOOM_INFERENCE_FEATURE_HEADER=X-Feature # optional, with the customer header: hyde, memory.search, worker.embedding
LOOM_INFERENCE_SYSTEM_CUSTOMER=loom # calls that belong to no org
LOOM_INFERENCE_CUSTOMER_REQUIRED=true # a cell serving several customers: refuse to start without the header
LOOM_INFERENCE_CUSTOMER_FORMAT='cloud:{org}' # the gateway's customer id for an org; default the org id
LOOM_INFERENCE_FEATURE_MAP_JSON='{"hyde": "search", "*": "memory"}' # Loom's feature names to the gateway's
- A request names the org it addresses, as authorized for its token; a caller cannot name an org its token does not reach.
- A background job names the org of the rows it works on, and makes one call
per org: the embedding worker,
loom reembedand a model rebuild split their batches by org. - The deployment’s own calls (the startup check of the embedder, the
recalibration worker’s fixed pairs, the dashboard’s provider check) name
LOOM_INFERENCE_SYSTEM_CUSTOMER. - The sampled retrieval judge (
LOOM_RETRIEVAL_JUDGE_SAMPLE_RATE) sends the judged org’s query and memories, so each judged search adds a call that names that org. - With the customer header set, a model call that names no org is refused
(
MODEL_CALL_UNATTRIBUTED) instead of being sent: it is a call path that forgot its org, and the gateway would bill it to no one. - An org id is sent percent-encoded where it needs to be.
Cached answers make no call: an embedding or a model answer served from Loom’s cache is not metered again. The customer is still checked before the cache, so a path that names none is refused even on a hit.
Naming the agent on every call
A gateway that bills or limits each agent reads whom a call is for from headers. Map each header to a template, and every chat, embedding, rerank and classify request to the gateway carries it:
LOOM_INFERENCE_IDENTITY_HEADERS_JSON='{"X-Tenant": "{tenant}", "X-Agent": "{agent}", "X-Actor": "{email}"}'
LOOM_INFERENCE_GATEWAY_HOSTS=gateway.example # required: identity headers go only to these hosts
LOOM_INFERENCE_TENANT_CLAIM=tenant_id # the workload trust claim {tenant} names (the default)
-
The placeholders are
{org},{namespace},{subject},{user_id},{agent}(the agent of a workload trust’s token,sa:<trust>:<agent>),{tenant}(the value that trust requires of itsLOOM_INFERENCE_TENANT_CLAIMclaim) and{email}(a signed-in person’s address). -
A header whose template renders empty is not sent: with bare placeholders, a call with no agent behind it (a person’s, or the deployment’s own) carries no agent header. The headers naming
{agent}or{tenant}go together or not at all, so an agent never goes without its tenant. -
A call that names no org is refused (
MODEL_CALL_UNATTRIBUTED) unless it is the deployment’s own: runloomcommands that call models with--org, and with--namespacewhile the headers name the agent (one with no namespace is refused rather than billed to the gateway’s default agent). -
Background jobs name the identity of the rows they work on: the writer of a memory or of a deferred extraction, else the agent whose own namespace the row is in. A job batching rows makes one call per identity.
-
No other host gets the identity headers: a reranker, a classifier or a provider outside
LOOM_INFERENCE_GATEWAY_HOSTSnever learns whom a call is for. -
LOOM_INFERENCE_IDENTITY_REFUSALS_JSONlists the answers your gateway gives for one identity, each by its status and the exactcode,typeormessageof its error body (undererror). Match bycodewhere the gateway gives one. The default matches an inference proxy’s per-agent codes and the budget refusal of the metering behind it:{ "refused": [ { "status": 429, "code": "budget_exceeded" }, { "status": 429, "type": "rate_limit_error", "message": "budget exceeded" }, { "status": 403, "code": "agent_denied" }, { "status": 400, "code": "identity_incomplete" }, { "status": 400, "code": "identity_malformed" } ], "wait": [{ "status": 429, "code": "agent_request_limit" }] }The second entry is matched by type and message because the metering’s Anthropic-shaped answer on
/v1/messagescarries no code. -
Never list an answer the gateway also gives for its own failure:
token_unavailable(it cannot fetch a credential, which a credential-service outage also answers) oridentity_missing(the request named no agent and the gateway has no default, a configuration problem). Listed,/v1/healthwould stop seeing the outage or the misconfiguration. Deployment-level answers stay provider failures too:caller_*,scope_not_granted(a route the token’s scopes do not cover),route_not_found,upstream_*,misconfigured,no_identity. -
An answer under
refusedmakes that request answer 403MODEL_CALL_REFUSED, and one underwait503MODEL_CALL_DEFERREDwith the gateway’sRetry-After; a wait of two seconds or less is waited out once first, and the call asked again. Nothing retries either answer,/v1/healthdoes not read them as a provider failure, and background jobs hold that identity’s calls to the same model, forLOOM_INFERENCE_REFUSAL_COOLDOWN_SECONDS(10 minutes by default) after a refusal and theRetry-Afterafter a wait, while other work goes on. Any answer the table does not list is the provider’s, as always. -
How each value is found: identity headers.
Switching an org’s model calls off
An org admin can stop every model call for the org’s data, with or without a gateway, from the dashboard’s Guardrails page or over the API:
curl -s -X POST "$LOOM_BASE_URL/v1/org.set_model_calls" -H "Authorization: Bearer $ORG_ADMIN_TOKEN" \
-d '{"org":"acme","enabled":false,"reason":"customer turned AI off"}'
- While off, no chat, embedding, rerank or classify request is sent for the
org, and no cached vector or answer is served to it either: the switch is a
kill switch. A request that needs one is refused (403
MODEL_CALLS_DISABLED):memory.setof a memory that is embedded on write, or in a namespace with a classify endpoint,memory.search, andtracker.similar,tracker.duplicate_checkandcode.bisect_candidateswhen they embed, judge or rerank, among them. A read by key (memory.get) and a delete make no model call and answer. The local embedder runs in-process and sends nothing, but it is switched off too. A call that names the org is checked even when Loom makes it for itself; only its own calls that name no org (the startup check, calibration) go on. "enabled": trueswitches it back on, and the waiting work runs.- A platform admin’s lock. A platform admin can add
"locked": true: the org’s own admins can then still switch it off, but switching it back on or unlocking it is refused (403, naming who locked it) until a platform admin sends"locked": false. A platform admin’s change that sends nolockedkeeps the lock.org.model_callsreads the setting, the lock and who set it. - Offboarding an org is manual: a platform admin sends
"enabled": false, "locked": true. The org’s cached query vectors expire after 30 days. No Loom path switches an org off by itself. - Every server applies a change within
LOOM_MODEL_CALLS_POLICY_REFRESH_SECONDS(default 10; the process that took it, on its next call). A server that cannot read the switch for longer thanLOOM_MODEL_CALLS_POLICY_MAX_STALE_SECONDS(default 60, 15 to 300) makes no model call for any org until it can. - Switch an org off only once every replica runs a build that knows the switch: an older one keeps calling models for the org.
What waits, and for how long, while an org is off:
| Work | While the org is off |
|---|---|
| Memory embeddings (the embedding worker) | Not picked: the memories stay, without a vector, and are embedded once the org is back on. Never dropped. |
| Extraction requests (the extraction queue) | Stay pending, unclaimed. Once a request has waited LOOM_MODEL_CALLS_OFF_MAX_WAIT_DAYS (default 7, 1 to 29) since it was queued and since the org was switched off, it is marked failed with a memory_drops record, reason model_calls_off. The record is written synchronously first: a request whose record cannot be written stays pending. The limit stays under 30 days because the queue’s 30-day TTL would remove the request first, with no record. |
| Entity tagging, profile folds, period digests, conversation titles, reflections, observation distillation | Not selected; done once the org is back on. They are derived from rows that stay, so nothing is dropped. |
| Projection repairs | Not scanned; retried once the org is back on. |
| Pending conflict judgements | Not re-judged. One older than the sweep’s hour when the org comes back is past the sweep, as a judgement lost in an outage is: the write stands as made. |
| Code ingest, code summaries, code embedding recovery | The org’s repositories are skipped, so its whole code index, lexical search included, stays as it was. A stub the recovery scan moved past is found by its periodic full scan. |
| Code-link judgements | Left due. |
| Tracker embeddings, tracker link scoring | The namespace is skipped before its cursor moves. |
| Retrieval-judge samples | Dropped: a sample is not owed work. |
loom reembed |
The org’s rows are left out, counted (skipped_switched_off) and reported; run it again once the org is back on. --delete-old keeps the old vectors of every org whose rows it left. |
| A model change’s rebuild | Completes; the org’s rows are staged without a vector and counted in the rebuild’s progress (rows_waiting). Once the org is back on, memory embeddings, code chunks and code summaries are embedded again by their recovery workers; a row of any other table has no vector until it is written again or a later rebuild covers it. |
Without a workload token
Where the platform gives Loom no token (a VM, a container host), Loom signs its own: set an audience instead of the token file, and the gateway trusts Loom as an issuer.
LOOM_INFERENCE_TOKEN_URL=https://gateway.example/oauth/token
LOOM_INFERENCE_GATEWAY_HOSTS=gateway.example
LOOM_INFERENCE_TOKEN_AUDIENCE=loom-inference # what the gateway's entry for Loom names
LOOM_PUBLIC_BASE_URL=https://loom.example # the issuer is <this>/v1
LOOM_OPENAI_BASE_URL=https://gateway.example/v1
LOOM_ANTHROPIC_BASE_URL=https://gateway.example
LOOM_LLM_PROVIDER=openai
LOOM_LLM_MODEL=gpt-6-luna
LOOM_EMBEDDING_MODEL=text-embedding-3-large
Each exchange sends a new token that lives five minutes: issuer
<LOOM_PUBLIC_BASE_URL>/v1, subject loom, the audience above, signed RS256
with the newest RS256 key of the signing keyring. Create one first with loom signing-key rotate --algorithm RS256, unless LOOM_JWT_SECRET is an RSA key,
which signs as RS256 already. That key signs Loom’s own tokens too from then
on, which every Loom of the deployment verifies; an HS256 secret is never used,
since the gateway could not verify it. Spend is attributed to that one subject.
The gateway’s entry for Loom names that issuer, the audience, subject loom,
RS256, and the keys at <LOOM_PUBLIC_BASE_URL>/v1/.well-known/jwks.json (OIDC
discovery is at /v1/.well-known/openid-configuration). The gateway must be
able to fetch that URL over https; where it cannot reach Loom, publish a copy of
the key set somewhere it can. Each Loom publishes a key at its next keyring
refresh after rotate writes it, and the key signs only --activate-in later
(300 s by default): rotate with an --activate-in longer than the gateway
keeps its copy of the keys, or than copying them takes. A retired key stays
trusted at the gateway until its copy is refreshed. loom signing-key retire
refuses to retire the last active RS256 key when LOOM_INFERENCE_TOKEN_AUDIENCE
is set in its own environment, so run it with the servers’. Loom (the server
and every loom command) refuses these settings without an https
LOOM_PUBLIC_BASE_URL, with LOOM_JWT_AUDIENCE as the audience, or with an
issuer its own verifier trusts, and its own verifier never accepts a token with
the gateway’s audience.
Apply changes and check the result
Dashboard changes: wait for Models & keys to report the active configuration. If a rebuild is running, follow its progress there; retry a failed job from the same page. Do not recreate containers to apply a dashboard change.
Environment changes only: for the released stack, recreate the server container:
docker compose --project-directory "$HOME/.loom" up -d --force-recreate loom
docker compose --project-directory "$HOME/.loom" exec loom loom doctor
curl -fsS http://localhost:7777/v1/health
Use your installation directory and port if different. Recreate the container: a plain
docker compose restart does not reload its environment from .env. Native servers
must restart with the updated environment. For the checkout stack, run
docker compose --env-file .env -f compose.yaml -f dev/docker-compose.dev.yml --profile bundled up -d --build --force-recreate loom.
Health reports fake_llm, fake_embedding, providers_degraded and actionable
problems. With both real providers configured, the fake flags should be false. This
checks configuration and health; also save and retrieve a test fact and try an answer to
confirm that your account can call the chosen models.
Changing the LLM or rotating an API key does not require rebuilding embeddings.
Changing the embedding model does, even when its dimensions stay the same. For
environment changes (rather than the dashboard workflow), existing
memory, document, tracker and code indexes need their corresponding re-embedding or
re-ingest procedures; loom reembed alone is not a universal rebuild of every index.
If dimensions change, check the existing schema’s vector constraints before applying
the new configuration. Preserve your data and follow
maintenance and
code-index operations.
Diagnose silent model or provider failures
Read the response body, not just the HTTP status:
curl -sS "$LOOM_BASE_URL/v1/health" | jq '{status, fake_llm, fake_embedding, providers_degraded, providers, problems}'
The HTTP status checks ClickHouse availability; it does not prove model access.
fake_llm and fake_embedding expose configured fakes or missing-key fallbacks.
They can both be false while a real provider fails: inspect providers_degraded, each
provider’s failing and last_error, and the fixes in problems.
Provider outcomes are passive, process-local observations of actual calls; polling
health makes no model request. No recorded outcome is not proof of working inference.
A later recorded success clears that provider’s failure. The compatible embedding
provider does not populate providers, so providers_degraded: false does not validate
that endpoint. Check its request errors and exercise retrieval. After checking configuration,
use the end-to-end smoke and try a generated
answer to exercise both providers. The smoke makes model calls and can incur charges.
For a stored memory missing from search, also check the embedding backlog described there.