Greg Low continues a series on local text embeddings:
In the first article of this series, I explained how to install and configure Ollama to host text embeddings models locally. I also demonstrated how to install Caddy as a proxy to allow SQL Server to use Ollama via https-based calls. In this article, I’ll show you how to make use of this at the SQL Server end.
Greg mentions a few embedding models, but the one I’m pushing right now is Microsoft Research’s Harrier OSS v1 model, specifically the 600m parameter version. It does extremely well in the MTEB leaderboards (the 27b variant is at the top of the board as of June 2026) and has a permissive license. It generates vectors in 1024 dimensions, so the embeddings are a bit chunky, taking up 4kb apiece. But the results are really good.