Skip to content
HumanChain

Bring-your-own-LLM

When no human answers in time, a consult falls back to an LLM. By default that’s HumanChain’s model. With bring-your-own-LLM, you point that fallback at your own OpenAI-compatible model instead - your provider, your key, your choice of model - while keeping everything else about the consult the same.

Why use it

  • You already have a model/provider you trust (or are contractually required to use), and you want fallbacks to go through it.
  • You want fallback answers to run under your own data and billing arrangement.
  • You want a stronger (or cheaper, or domain-tuned) fallback than the default.

Either way, the honesty contract holds: a fallback from your model is labeled source=client_llm in the transcript, distinct from both a human answer and HumanChain’s default llm_fallback. The agent always knows what it’s reading.

Configure it

Master authentication required. The key is encrypted at rest and is never returned by the API.

Terminal window
# Set or replace your fallback model
curl -X PUT \
  https://hub.example.com/api/master/llm-config \
  -H "Authorization: Bearer <your master key>" \
  -H "Content-Type: application/json" \
  -d '{
    "base_url": "https://api.openai.com/v1",
    "model": "gpt-4o-mini",
    "api_key": "sk-…",
    "label": "our OpenAI account",
    "enabled": true
  }'


# View it (the api_key is NOT included in the response)
curl https://hub.example.com/api/master/llm-config \
  -H "Authorization: Bearer <your master key>"


# Remove it (revert to HumanChain's default fallback)
curl -X DELETE https://hub.example.com/api/master/llm-config \
  -H "Authorization: Bearer <your master key>"
FieldWhat it is
base_urlAn OpenAI-compatible API base (OpenAI, Azure OpenAI, Groq, Together, a self-hosted vLLM, etc.).
modelThe model name to call at that endpoint.
api_keyYour provider key. Encrypted at rest; never returned.
labelAn optional human-readable name for your own reference.
enabledToggle the config on/off without deleting it.

How it behaves on a consult

When a consult would fall back to an LLM and you have an enabled config, the Hub calls your model and labels the answer source=client_llm. If your model errors or times out, the Hub gracefully falls through to its own default so a consult always returns something, honestly labeled - your fallback being down never breaks a consult.