ENDE

I tested 13 local AI models on Reflacto's real prompts

Which local model should Reflacto recommend? I measured 13 of them with the prompts the app really sends: do they follow the instructions, and do their answers help?

Daniel Schmeiß

Reflacto's AI features only work with a model that runs on your own computer, so your reflection stays on your device. While you write, Reflacto can ask you a question, point out what you might be missing, suggest ideas or help you find the right words. At the end it writes a short action plan.

So which local model should I recommend? A big model will not run on a normal laptop. A small one often ignores the instructions. I spent a Saturday measuring it.

The short version

  • Almost every model follows Reflacto's instructions. The best nine are too close to name a winner.
  • Whether an answer helps is a different question, with a different winner. The three big models gave clearly more helpful answers than the small ones.
  • Gemini Nano, already built into Chrome, follows the rules too. Its answers help less than those of Gemma 4 E4B.
  • The biggest jump came from my own prompt. One unclear rule made good models look bad.
  • For laptops I recommend Gemma 4 E4B, with a 16 GB graphics card Gemma 4 26B.

How I measured

I wrote 12 made-up situations in German and English. In one of them, a tech lead called Riley keeps interrupting the product managers in planning meetings. I sent the real prompts of Reflacto to every model, 96 requests per model.

Each answer went through 8 automatic checks. Is it in the right language? Does it have the right form? Does it stay within the word limit? Does it quote only what the person wrote?

These checks show if a model follows Reflacto's prompt. They do not show if the answer helps. For that I asked Claude to rate every answer.

The test setup

I built the test with Evalite opens in a new tab, a free tool for testing AI features in TypeScript (source code opens in a new tab).

  • The test uses Reflacto's own code. It builds each prompt with the same function as the app and sends it through the same service. When I change a prompt in Reflacto, the next test run uses the new version.
  • The 8 checks are plain TypeScript. No AI decides whether an answer passes, so the same answer always gets the same result.
  • 11 models ran on my desktop PC with a GeForce RTX 5070 Ti (16 GB), served by Unsloth Studio. gemma3:1b ran in Ollama on my Mac (M2, 16 GB RAM), Gemini Nano in Chrome on the same Mac. Gemma 4 E4B ran there a second time, to see how it does on a smaller computer.
  • Requests ran one at a time, so no model had to share the graphics card.
  • Evalite keeps every answer next to its result and shows them in a local web page. Behind each number I could read the answers that produced it.
  • For the content scores, Claude saw all answers to one prompt side by side, shuffled and without model names.

Almost every model follows the rules

Share of checks passed. The line around each dot is the margin of error. Where lines overlap, the order is not reliable.

The best nine models pass between 98.4% and 99.6% of the checks. Their lines overlap, like the margin of error in a poll. With a few other requests, the order would be different.

Three models stay clearly behind: two small ones, LFM3-1.7B and Granite Switch 8B, and Qwen3.8 in a strongly compressed version.

Not every answer helps

Claude read all 1,247 answers and rated each from 1 to 10. Does it move the person forward? Does it stay with what they wrote? It saw the answers side by side, without model names.

Here is an example. In the Riley case, the person wrote:

In the last two sessions Riley began answering before questions were finished and redirected three agenda items to implementation details.

The help "Facts or interpretation?" should point out where this is opinion instead of observation. Gemma 4 E4B in 4 bit answered:

"began answering before questions were finished" This is a judgment about Riley's actions. What exactly did Riley do when they began answering before questions were finished, and when did this happen?

But that is what a camera would have recorded. The person did it right, and the answer says they did it wrong. Gemma 4 26B answered:

Your description already focuses on what happened. To make this even more specific, could you describe exactly when or where these redirections occurred during the sessions?

Both answers passed all 8 checks. Claude gave the first a 3 and the second an 8.

Share of poor, middling and good answers per model. The number on the right is the average content score from 1 to 10.

This pattern holds across all answers. The three big models, Gemma 4 26B, Qwen3.8-27B and Bonsai 2, score about 6.5. More than half of their answers are good. Gemma 4 E4B scores 5.5. Only one answer in five is good. It keeps the form, but often stays generic.

Two more results stood out. gemma3:1b passes 96.9% of the checks, but not one of its answers was good. And gpt-oss-20b made up facts in 10 answers.

And the model that is already in Chrome?

Chrome comes with its own small model, Gemini Nano. Reflacto can use it without installing anything. Chrome gives web pages access to it through the Prompt API opens in a new tab.

Reflacto calls it directly in JavaScript. LanguageModel.create() starts a session with the instructions, and promptStreaming() sends the person's text and returns the answer word by word. After each answer Reflacto closes the session again. The model runs inside Chrome on your computer, so no server is involved and the text never leaves the browser. The architecture page explains how Reflacto connects to Chrome, Ollama and other local models.

Gemini Nano passes 98.6% of the checks, level with the top. Its content score is 4.9, clearly behind Gemma 4 E4B and level with Bonsai-27B. In 7 answers it made up facts.

The biggest jump came from my own prompt

Answers in the right form for “What am I missing?”, out of 20, before and after the prompt change.

In the first run, Gemma 4 E4B gave 0 of 20 answers in the right form for "What am I missing?".

When I read them, they were good. The check expected bullets, and Gemma wrote one short paragraph. Then I read my prompt again. It asked for bullets, but one rule said "If their answer is already solid, say so in one sentence". Gemma followed that rule.

I made the prompt clear. After that, 7 of 12 models got 20 of 20.

When several good models fail at the same place, I now read my prompt first.

Smaller files cost quality, but not always

Many models come in compressed versions, a bit like saving a photo as a smaller JPEG. The file gets smaller and the model faster.

The same model in different versions.

For Gemma 4 E4B, 4 bit instead of 8 bit cost nothing I could measure. At 2 bit it still passes the checks, but its answers get clearly worse.

Qwen3.8-27B in the smallest version falls from 99.3% to 85.2%. It even returns empty answers. The file is half the size, but the model is only a third faster. Not worth it.

Speed, length and temperature

Tokens per second on the PC. Gemini Nano and gemma3:1b ran on the Mac and are not included.

gpt-oss-20b is the fastest of the larger models. Qwen3.8-27B is the slowest, but still needs only about 2 seconds for a short answer.

Each dot is one answer, its length compared to the word limit. The number on the right is the share of answers that were too long.

Qwen3.8, Gemma 4 26B and Gemma 4 E4B never went over the word limit. Granite Switch 8B was too long in a third of its answers.

Share of checks passed per temperature setting.

The temperature controls how much a model varies its words. For the two best models it changed almost nothing. Prompt and compression mattered much more.

Things that went wrong

The benchmark found a bug in Reflacto. When you switched to another version of the same model, Reflacto kept the old one loaded. That is fixed now.

And my PC crashed once, right when one big model was replaced by the next. After that I ran the models in groups.

What I recommend for Reflacto

For laptops: Gemma 4 E4B in 4 bit. It follows the rules as well as any model here, and its answers are fine, just not as good as those of the big models. On my Mac (M2, 16 GB RAM) it passed 99.6% of the checks, as on the PC, and its answers were rated about the same. It is slower there: a short answer while you write takes about 6 seconds, the action plan about 19. The Ollama version needs 9.5 GB instead of 5.1 GB, because it also contains the parts for images and audio.

With a 16 GB graphics card: Gemma 4 26B. It gave the most helpful answers and is almost as fast. Qwen3.8-27B is just as good if you can wait a few seconds. I no longer recommend gpt-oss-20b, because it makes up facts too often.

Without installing anything: Gemini Nano in Chrome. It works right away, but expect simpler answers.

The local AI guide walks you through setting up one of these models for Reflacto, step by step.

What these numbers do not say

Every model ran once, so the real uncertainty is larger than the charts show. The content scores come from one AI judge. Claude is not a person stuck in a hard conversation.

Try it yourself

Reflacto is free, needs no account and runs in your browser. Your reflection stays on your device.

Start a reflection