# Building an AI chat assistant for my site, part 2

In part 1 I laid out the plan: a RAG-based chat assistant for this site, grounded in my blog posts and a profile I'll build about myself. Before writing any real application code, I said there was one question that could reshape the whole design: would embeddings I compute locally, through Ollama on my Intel GPU via IPEX, actually match the same model hosted on Cloudflare?

The architecture depends on this working. The plan is to embed content locally when I author it. It's free and private, but it also simplifies things: generating the embedding and inserting it into Cosmos DB both happen in one local step, no round trip to Cloudflare needed just to write a new article or profile fact. A visitor's question is different though. That has to be embedded through Cloudflare Workers AI at chat time, since it has to run whenever someone visits the site, not just when my machine happens to be on. If those two don't produce compatible vectors, the whole split falls apart, and every embedding, not just the live ones, would have to go through Cloudflare instead.

So I tested it directly. Same model both sides, `bge-m3`, chosen because it's multilingual and this site needs to work in both English and Dutch. First, the local side, through Ollama:

```powershell
$body = @{ model = "bge-m3"; input = "I have experience running Kubernetes clusters on Azure." } | ConvertTo-Json
$local = Invoke-RestMethod -Uri "http://localhost:11434/api/embed" -Method Post -Body $body -ContentType "application/json"
$local | ConvertTo-Json -Depth 10 | Out-File local_vector.json
```

Then the same sentence through Cloudflare's Workers AI REST API:

```powershell
$headers = @{ Authorization = "Bearer $token" }
$body = @{ text = @("I have experience running Kubernetes clusters on Azure.") } | ConvertTo-Json
$cf = Invoke-RestMethod -Uri "https://api.cloudflare.com/client/v4/accounts/$accountId/ai/run/@cf/baai/bge-m3" -Method Post -Headers $headers -Body $body -ContentType "application/json"
$cf | ConvertTo-Json -Depth 10 | Out-File workers_ai_vector.json
```

I saved both responses to files and had Claude write the actual comparison, extracting the two vectors and computing cosine similarity between them, not something I wanted to do by hand. Along the way Claude also caught that both files needed decoding as UTF-16 rather than UTF-8, since that's what PowerShell's `Out-File` defaults to on Windows, and noticed Cloudflare's response carries extra metadata Ollama's doesn't, which is why one file looked almost twice the size of the other before either of us had actually looked inside it.

First test sentence, the one above, in English: 0.999933. Second, the same test in Dutch ("Mijn naam is Patrick Koorevaar en mijn hobbies zijn: hardlopen, technische artikelen maken en het bijhouden van ontwikkelingen op Linkedin."): 0.999934. Both essentially identical, both landing in the same 1024-dimensional space.

One detail from Cloudflare's own response turned out to matter more than I expected: it explicitly reports which pooling strategy it uses, CLS-token pooling. That's exactly the kind of implementation detail that could have silently broken this test even with the "same" model, if Ollama's runtime happened to use a different strategy underneath. It didn't, but it was worth knowing to look for.

Smaller, funnier finding along the way: one of the two response files looked almost twice the size of the other, which briefly looked like a red flag. Turned out to be PowerShell's `Out-File` defaulting to UTF-16 encoding on Windows, nothing to do with the actual data.

With this confirmed, the plan holds: local authoring, Cloudflare serving. Next up is actually building the knowledge base these embeddings will live in.

---

*Co-authored with Claude.*

If this was useful, [you can buy me a coffee](https://ko-fi.com/p47k0).

New articles: [follow via RSS](https://blog.koorevaar.com/feed.xml).