Choosing a model

Short version: Engram needs no AI model to be useful. The thing you came for — asking a question in plain language and getting back the few pages that actually answer it — is pure retrieval. No model, no API key, no GPU. It works the moment you install it.

A model only makes one part nicer: turning a finished session into a well-written page. So the question on this page isn't "do I need a model to use Engram" (you don't) — it's "how good do I want the automatic write-ups to be, and what am I willing to set up for that."

What a model actually does here

Two things, both at the end of a session, both in the background, neither on the path you wait for:

• Distilling a session into a page. With no model, Engram writes a clean, structured outline of the conversation — the questions asked and the answers reached. With a model, it writes a flowing prose summary instead.
• Splitting a big page into focused ones. With no model, a page stays whole. With a model, an overgrown page can be broken into tidy, single-idea pages that recall more precisely.

Because both happen after you've stopped working, a slow model is perfectly fine — you're never staring at a spinner waiting for it.

The three tiers

Tier 1 — No model (the default)

Nothing to install. Digests are structured outlines; recall is full-strength. This is the honest baseline, and for a lot of people it's all they'll ever want. Cost: zero. Privacy: total — nothing ever leaves your machine.

Tier 2 — A small local model (recommended if you want nicer write-ups)

Run a small model on your own machine — through Ollama or a compact GGUF file — and Engram uses it for distillation while your data never leaves the box. This is the sweet spot: private like Tier 1, but with genuinely nicer pages.

Good choices, all small enough for a laptop:

You don't need a powerful GPU. Since distillation runs after the session, even CPU-only is workable — it just takes a little longer in the background. If you have any modern discrete GPU, it's quick.

Tier 3 — A cloud model (best quality, needs a key)

Point Engram at a hosted model (via Ollama's cloud, or another provider) with an API key. You get the richest summaries and pay per use, and — unlike the first two tiers — the session text is sent to that provider to be summarized. Choose this if write-up quality matters more to you than keeping everything local.

Which should I pick?

• Just want great recall, zero fuss? Stay on the default. You're not missing the main event.
• Want nicer automatic pages, and care about privacy? A small local model (Tier 2) is the one to reach for — it's the best balance for most people.
• Want the very best summaries and don't mind a key + sending text to a provider? Go cloud (Tier 3).

Whatever you pick, you can change your mind later, and recall keeps working exactly the same underneath.

A note on honesty, since it's the whole spirit of the project: the numbers we publish for recall — how much less it reads than a blind search — are measured with no model at all. The model is a convenience for writing, never a crutch the core depends on.