Blog
Home

Building an AI chat assistant for my site, part 6

13 September 2026 · 65 views

Part 5 ended on an honest note: the assistant worked, but asked what I do for work, it gave a thin answer, because the knowledge store only held three test articles. The obvious next step was to stop testing with three articles and actually feed it everything.

That meant two very different kinds of content. The first was mechanical: embed all 93 blog articles, the same way the first three were done, one summary chunk per article plus one chunk per paragraph. A script, written and run through Claude Code, fetched every article from the write API, split each one into paragraphs on blank lines, embedded each chunk locally through Ollama, and wrote it into the knowledge store, replacing by a predictable id if it already existed so the whole thing could be re-run safely. It ran for about half an hour and produced just over a thousand chunks, with a single transient timeout that a retry cleared.

The second kind of content couldn't be mechanical at all. The knowledge store had nothing about me personally beyond what the blog happened to mention in passing, no current role, no certifications list, no real skills summary. So I went looking: my LinkedIn profile, the actual content already sitting on my own site's homepage, and a fairly long back-and-forth where I filled in the rest myself, hobbies, the reason I keep building side projects, what I'm actually looking for professionally. Claude drafted candidate facts from what it could find and read back to me, and I corrected them as we went. One real mistake got caught this way: an early draft had merged two separate side projects, a Kubernetes automation script and a browser-based practice lab, into one fact, simply because they were both about Kubernetes. Reading the real site content split them back into two.

With both kinds of content in place, the same test question from part 5 came back completely different: instead of a vague guess about writing and blogging, it correctly described real work on cloud-native Azure infrastructure, the blog platform, and the side projects, pulled straight from the new facts.

The last thing I tested wasn't retrieval at all, it was manners. A plain "hi" was hitting the exact same refusal path as a genuinely unanswerable question, which reads as broken for a chat widget. So did a "bye." So did an off-topic question about the weather. Three small talk cases went into the guardrail prompt: greetings get a warm invitation to ask something real, farewells get a goodbye and nothing else, fully unrelated questions get a light redirect back to Patrick instead of a formal refusal. Two landed clean on the first try. The third didn't: an English goodbye kept coming back with a Dutch flourish, and a Dutch goodbye was sometimes read as a Dutch hello, since the word for both is the same. My first fix, removing the Dutch example words from the prompt on the theory that they were leaking into the model's language choice, made no difference at all on retest. Rather than keep guessing at prompt wording, I'm leaving it as a known limitation of an 8B model doing bilingual intent detection under time pressure. The replies are still warm and on topic either way, just occasionally bilingual when they shouldn't be.

Content is now in reasonable shape. What's left is entirely plumbing: keeping the cache honest when an article changes, protecting a paid LLM call from abuse, and finally building something a visitor can actually click on.

Co-authored with Claude.

If this was useful, you can buy me a coffee.

New articles: follow via RSS.

Comments

← All articlesView as Markdown