Category: Vectors

Greg Low continues a series on local text embeddings:

In the first article of this series, I explained how to install and configure Ollama to host text embeddings models locally. I also demonstrated how to install Caddy as a proxy to allow SQL Server to use Ollama via https-based calls. In this article, I’ll show you how to make use of this at the SQL Server end.

Greg mentions a few embedding models, but the one I’m pushing right now is Microsoft Research’s Harrier OSS v1 model, specifically the 600m parameter version. It does extremely well in the MTEB leaderboards (the 27b variant is at the top of the board as of June 2026) and has a permissive license. It generates vectors in 1024 dimensions, so the embeddings are a bit chunky, taking up 4kb apiece. But the results are really good.

Greg Low has a demo:

Instead of working directly with text, images, or other rich content, an embedding represents that content as a set of numbers that capture semantic relationships learned by a model. This lets systems work with meaning in a mathematical way rather than relying on concepts like text matching.

Embeddings are the output of trained text-based AI models. One possibly surprising concept is that they’re used for similarity, as opposed to facts – and for relative closeness instead of exact matches.

SQL Server is not designed to host or execute those models. This isn’t a limitation; it is a design choice. While it would be possible to run code within SQL Server to generate embeddings, it just wouldn’t be a good idea.

Note that SQL Server can already run machine learning models directly, and use the PREDICT statement to make predictions. We don’t want to be doing that with the language models we need for embeddings, though.

There are also some really good local embedding models, and between that and the vector similarity functionality in SQL Server, you can perform semantic search without needing to reach out to an online service.

Andrew Pruski does a bit of math:

Here we are using cosine as a distance metric, but there are three available to us in SQL Server: –

• Euclidean Distance
• Negative Dot Product
• Cosine Distance

In this post we’ll run through all three, see how they are calculated, and explore how they differ from each other.

Click through for a description of each.

Greg Low breaks down a document:

If you’ve started to work with vector databases and looked at using text embeddings for AI search, you might have come across the term chunking and wondered what it relates to. In this article, I’ll explain the concept in general – and then show how it works in SQL Server 2025.

Read on for that explanation. Greg also includes a quick example of how this looks in SQL Server 2025 when passing text data through an embedding model.