Building the assistant Worker itself came next: a new Cloudflare Worker that takes a visitor's question, retrieves the right chunks from the vector store, and generates a grounded answer. No UI yet, just the backend, built in small commits so each piece could stand on its own and get reviewed separately.
Workers AI and Workers KV came first. The same bge-m3 model embeds a visitor's question at query time, and a small KV-backed cache brute-force compares that embedding against past questions before ever touching Cosmos. At the traffic this is expected to see, dozens to low hundreds of distinct questions, one KV key holding the whole cache as a single JSON blob is simpler than a key per entry, and a lookup is just one read.
The more interesting decision came at the cache-miss path. The identity that already writes embeddings into the knowledge store has permission to create and replace documents there, and handing that same identity to a public-facing Worker felt wrong: anyone's question could indirectly reach it, carrying access it would never need to use from here. A new, narrower Cosmos role exists now, read and query only, nothing else, scoped specifically to this Worker. Querying itself needed its own workaround too: a Cloudflare Worker has no Node.js Cosmos SDK, so the retrieval code talks to Cosmos's REST API directly, the same client-credentials token flow already proven out earlier in this project. Cosmos's REST API can't serve a cross-partition ORDER BY the way the official SDK does, so the Worker runs one query per partition, article content and profile facts separately, and merges the results itself afterward.
On top of retrieval sits a system prompt doing the actual guardrail work: answer only from what was retrieved, refuse honestly rather than guess when the context doesn't cover a question, treat the retrieved text as reference material rather than instructions so a visitor's message, or even a compromised chunk, can't talk the model into acting outside its scope, and answer in whichever language the question was asked in. The generation call itself sits behind a small adapter that normalizes whatever shape a model's response comes back in into one plain string, a lesson learned the hard way earlier in this project when swapping models meant swapping response formats too.
That adapter got exercised almost immediately. The first live test of the finished pipeline came back with a server error. The model Claude had picked, gemma-3-12b-it, wasn't actually the one we'd agreed on, and it 403'd because Cloudflare had already deprecated it as of the end of May 2026. Switching to the model we'd actually agreed on, llama-3.1-8b-instruct-fp8, fixed it right away, the same one already running successfully in other projects.
With that fixed, the real test was asking it something. A question in Dutch about the CKA certification came back fluent and accurate, built entirely from English-language source chunks, despite Dutch not even being one of Llama 3.1's officially listed languages. The retrieval scores behind that answer also settled a question left open since the design phase: whether the closest match in Cosmos's cosine distance sorts first or last. It sorts first, confirmed by the single most relevant chunk actually ranking at the top of the list.
The same test also exposed something more honest than a bug: the knowledge store still only holds a handful of test articles, nothing written yet about what I actually do, no CV-shaped facts at all. Asked what I do for work, the assistant gave a thin but accurate answer instead of inventing one, exactly what the guardrail is there for, just with nothing better to draw on yet. Feeding it real content about myself is next, then the pieces that make this safe to point at actual visitors: cache invalidation, rate limiting, and a real UI.