Model configuration

Hindsight never names a model server. It calls the LiteLLM proxy at http://litellm:4000/v1 with three fixed aliases, and the proxy’s YAML maps each alias to a model:

Alias	Hindsight operation
memory-retain	retain (extraction on ingest)
memory-reflect	reflect, the base HINDSIGHT_API_LLM_MODEL, and mental-model refresh. Needs a model that makes tool calls
memory-consolidate	consolidation

Shipped routes

Edit the YAML (or a per-machine override) to change models.

File	Backend	Default split
config/litellm.lemond.yaml	lemond on the host	Reflect: Gemma 4 E4B GGUF (unsloth/); retain and consolidate: E2B
config/litellm.ollama.yaml	Ollama on the host	Used when MEMORY_BACKEND=ollama; gemma4:e4b needs Ollama newer than 0.20.3

Why reflect uses a larger model: E2B often returns no usable tool call on Hindsight reflect (finish_reason=length). The shipped lemond route uses E4B for memory-reflect only. Qwen3.5-4B works as an alternative if you change the route; pick any model that passes a direct tool-call check and a smoke reflect.

Route selection

Priority	Path
1	~/.config/hindsight/litellm.yaml — full override, mounted read-only into the proxy
2	config/litellm.$MEMORY_BACKEND.yaml — lemond (default) or ollama

scripts/setup.mjs writes LITELLM_CONFIG in the stack .env. After changing backend, override, or routes:

node scripts/setup.mjs
docker compose up -d litellm

Optional Ollama backend:

MEMORY_BACKEND=ollama node scripts/setup.mjs
docker compose up -d litellm

Thinking and token limits

Thinking is off on all three aliases in the route YAML (enable_thinking: false / think: false) so replies are not empty when Hindsight sends low reasoning_effort.

Layer	Role
LiteLLM litellm_params.max_tokens	Default when the client omits max_tokens (4096 in shipped routes)
HINDSIGHT_API_*_MAX_COMPLETION_TOKENS in hindsight.env	Hard cap Hindsight enforces (retain 16384; reflect and consolidation 2048)

Retain cap must exceed each retain strategy’s chunk size (coding-agent strategies use 12000) or Hindsight returns 400 Invalid retain strategy.

Secrets

~/.config/hindsight/litellm.env (mode 600): LITELLM_MASTER_KEY (Hindsight uses this as HINDSIGHT_API_LLM_API_KEY), plus provider keys referenced as os.environ/NAME in the YAML.

Example hosted reflect model:

  - model_name: memory-reflect
    litellm_params:
      model: anthropic/claude-haiku-4-5
      api_key: os.environ/ANTHROPIC_API_KEY

Hindsight always sends the alias names above; you do not edit model ids in hindsight.env for route changes.

Verify and migrate

Health check from the compose network:

docker compose exec litellm python3 -c "import urllib.request;print(urllib.request.urlopen('http://127.0.0.1:4000/health/liveliness').read())"

Older hindsight.env files that pointed at lemond/Ollama directly:

node scripts/setup.mjs --migrate-env
docker compose up -d

Migration rewrites LLM lines to use http://litellm:4000/v1 and the three aliases; other lines stay as they are.

---

This page is generated from docs/model-configuration.md in the repository.