Use LiteLLM with Synth

LiteLLM exposes an OpenAI-compatible API, so configure it through Synth's existing openai provider. You do not need a separate LiteLLM provider.

You need three values:

The LiteLLM API URL, normally ending in /v1.
A LiteLLM API key stored in a Kubernetes Secret.
The model name configured in LiteLLM.

Understand The Model Name

Synth sends the model name to LiteLLM exactly as you configure it. Use the model_name from your LiteLLM configuration; do not invent a Synth-specific name.

For example, if LiteLLM contains:

model_list:
  - model_name: opencode
    litellm_params:
      model: your-provider/your-model

then configure defaultModel: opencode in Synth.

The model name does not come from the Kubernetes Secret. The Secret contains only the API key used to authenticate to LiteLLM.

Configure Helm

First, create or update a Kubernetes Secret. The key name is your choice; this example uses LITELLM_API_KEY:

kubectl -n synth create secret generic synth-runtime-secrets \
  --from-literal=LITELLM_API_KEY="$LITELLM_API_KEY" \
  --dry-run=client -o yaml | kubectl apply -f -

Then reference that Secret from your Synth Helm values:

synth:
  defaultProvider: openai
  defaultModel: opencode
  providers:
    openai:
      enabled: true
      baseUrl: https://litellm.example.com/v1
      defaultModel: opencode
      existingSecret: synth-runtime-secrets
      existingSecretKey: LITELLM_API_KEY

Replace:

https://litellm.example.com/v1 with the URL reachable from the Synth pods.
opencode with the exact LiteLLM model_name you want Synth to use.
LITELLM_API_KEY with the key name that already exists in your Kubernetes Secret. For example, if your Secret uses GABY_OPENCODE_API_KEY, set existingSecretKey: GABY_OPENCODE_API_KEY.

Do not put the API-key value in values.yaml. existingSecret is the Secret name, while existingSecretKey is the key inside that Secret.

After changing the provider Secret, restart the Synth web and ingestion-worker pods so both workloads receive the updated value.

Configure In The UI

You can configure the same gateway in Global Settings → Providers → OpenAI:

Set Base URL to the LiteLLM API URL, such as https://litellm.example.com/v1.
Set Default Model / Gateway Route to the LiteLLM model_name, such as opencode.
Save and select Test.

When Helm supplies the API key through existingSecret, Synth reads it as the OpenAI provider's runtime credential. You do not need to copy the key into the UI.

Use Different Models Per Agent

If all model-assisted jobs should use the same LiteLLM model, the provider's defaultModel is enough. To use different LiteLLM models, set each agent to a real model_name from your LiteLLM configuration:

synth:
  agents:
    extract:
      provider: openai
      model: opencode
    link:
      provider: openai
      model: opencode
    lint:
      provider: openai
      model: opencode

Change those values only when LiteLLM exposes different model names for those jobs.

URL Rules

Configure the API root, normally ending in /v1.
Do not append /chat/completions or /models.
Use an absolute http:// or https:// URL without embedded credentials, query parameters, or fragments.
From a pod, localhost means the Synth pod. Use the LiteLLM Kubernetes Service name or another hostname reachable from the pod.
Synth sends the configured LiteLLM model name unchanged.

Troubleshooting

Symptom	What to check
No credential configured	Confirm `existingSecret` names a Secret in the Synth namespace and `existingSecretKey` matches a key inside it. Restart the web and ingestion-worker pods after changing the Secret reference.
Test reports an unknown model	Confirm `defaultModel` exactly matches a LiteLLM `model_name`, including case.
`404` or a duplicated path	Set `baseUrl` to the API root, such as `https://host/v1`, not a completion endpoint.
Connection refused	Use a hostname reachable from the Synth pods instead of `localhost`.
Requests reach OpenAI directly	Confirm the effective `openai` provider has the LiteLLM `baseUrl` and that the pods were restarted after the configuration change.

LiteLLM remains responsible for choosing the upstream provider, enforcing budgets, and applying its own routing or fallback policy.

Understand The Model Name​

Configure Helm​

Configure In The UI​

Use Different Models Per Agent​

URL Rules​

Troubleshooting​