In the previous part of this series I described a weekly loop that improves a translation prompt from the corrections people make before publishing. Plain code counts the corrections, a larger model (gpt-oss-120b) proposes at most three new "learned rules", a filter and a test against frozen texts drop the bad ones, and what survives arrives as a pull request that I merge or close. At the time it had only run on made-up sample rows.
This post is about the first run on real logged corrections, and about the four changes to the loop it led to.
The data
No customer used these features yet, so the first real data came from me: I used both features on two demo websites and corrected the suggestions the way an owner would before publishing. That gave 30 rows: 20 translations (12 corrected) and 9 rewrites from the "improve this text" feature (3 corrected, 4 refused by the fact check).
A run cost about 900 to 1,100 neurons: 1,095 the first time, which also asked Llama 3.3 70B for comparison, and 918 the second. That is well inside the free plan's 10,000 a day.
A useless rule that passed the test
On both sites I had changed the Spanish "en la plancha" to "a la plancha" (on the griddle). The counting only reported the swapped words, clients replaced "en" with "a", and with no context gpt-oss-120b invented a grammar rule:
When expressing movement toward a place, use the preposition "a" instead of "en".
It sounds correct and fixes nothing. It still passed the test, because the English rule from the same run ("chapa" as "griddle", not "grill") brought the translations closer to what was published overall. The test judges a whole prompt file, not each rule, so a good rule carried a useless one through. The review caught it.
A placeholder that became a rule
The counting wrote an added word as replaced "(nothing)" with "o". The model read "(nothing)" as text and proposed removing placeholders such as "(nothing)" from rewrites. The test dropped that one, but only because the rewrites got worse for other reasons.
The fixes
So I closed the first PR of the scheduled run, made four changes to the loop, and ran it again by hand on the same rows. Each change answers one thing the first run got wrong:
| What went wrong | What I changed |
|---|---|
The model saw only "en" -> "a" and invented a grammar rule |
Context: every word change in the evidence now comes with the two words on each side |
| The model read "(nothing)" as text | Plain wording: an added or removed word is written as added "o" or removed "nosso", never "(nothing)" |
| The model wrote rules about bare words | One extra line for the proposing model: a rule must fit what the example phrase shows, not the bare words alone |
| A manual run could not re-read the same rows: a run only reads rows since the last successful run | A hand-run option to start reading from a given date |
The evidence lines now look like this:
T3: clients replaced "en" with "a", e.g. "se hace en la chapa" -> "se hace a la chapa" (into Spanish); 2 times on 2 sites
R1: clients added "o", e.g. "sua casa pagamento via" -> "sua casa o pagamento via"; 2 times on 2 sites
What did not change: the counting. A pattern is still grouped by the changed words only, and still needs two websites. The fixes are in what the model is shown, not in how strict the loop is.
The second run
The re-run read the same 30 rows, and the proposals became:
Translate the phrase "na chapa" as "a la plancha" (use the preposition "a" instead of "en").
Translate the phrase "na chapa" as "on the griddle" (use "griddle" instead of "grill").
No regression on the frozen texts, closer to what was published (word-by-word closeness 0.865 → 0.879), and merged: the translation prompt's first two learned rules.
For the rewrites, the counting now saw people adding "o" (the), as in "Pagamento via Pix" → "O pagamento por Pix". The model turned that into a broad "always use articles" rule, and the test refused it again (closeness 0.747 → 0.665).
A pattern I expected did not show up
I had removed "nosso" (our) from several rewrites, and expected the loop to notice. Word by word, though, those were different changes (nossa → a, nosso → o, nossos → um trailer de), almost all on one site, and on the other site I had kept "Nossa equipe". The loop counted nothing, and that was right: it was my taste, not a pattern across websites.
What I took from it
- What the proposing model is shown matters more than what it is told. A bare word swap invites a made-up rule; the same swap with a few words of context gave the right one.
- A test of the whole prompt can let a weak rule ride along with a good one. The human review is the last filter, and here it was needed.
- Corrections I make myself are a biased sample. They proved the pipeline; real owners will decide which rules it learns.
What I still don't know
- Whether the learned rules generalise. Both cover only the phrase "na chapa"; a menu that says "hambúrguer de chapa" is not covered.
- Whether the test should judge each proposed rule separately. That would have dropped the useless Spanish rule, at the cost of more neurons per run.
- Whether two websites is the right threshold once real owners correct texts, rather than one person on two demo sites.
Caveats
- Both runs read the same 30 rows, all from my own test corrections on two demo websites, in October 2026.
- Neuron numbers are measured on those runs; a week with more rows (the loop reads at most 300) will cost more.
- Prices and model availability are those of Workers AI in October 2026, on a free account.