Category: Generative AI

Erik Darling details a series of problems and somewhat-solutions:

This post is inspired by my BFFL Joe Sack’s wonderful post: Keep Humans in the Circle.

It’s an attempt to detail my progression using LLMs, up through how I’m using them today to build my free SQL Server Monitoring and Query Plan analysis tools.

While these days I do find LLMs (specifically Claude Code) to be wonderful enablers for my ideas, they still require quite a bit of guidance and QA, and they’re quite capable of (and sometimes seemingly eager to) wreck your day.

I did enjoy reading about Erik’s journey, so I figure I’ll share something of my own.

Jonathan Stewart (linking SQL Bites because I’m actively trying to shame him into creating a first video) has gone head-first into Claude Code and has dragged me along kicking and screaming. A lot of what Erik mentions resonates well with me, but there’s something that Jonathan developed that has helped: the Hater persona as an MCP server.

The Hater persona’s job is to critique whatever solution the language model comes up with. Find all of the nitpicks, point out the major gaps in implementation, come up with scenarios in which this just won’t work, that kind of thing. I’ve been Jonathan’s Hater-as-a-Service for years, so naturally, he named the MCP server Kevin. Artificial Kevin can iteratively to come up with the biggest problems, and feed that information back into the main model to fix it up. After several rounds of this, I’ve found that there aren’t nearly as many rough edges as you might find at the start.

Even so, I still stand by the assertion that language models are akin to drunken interns, and the extent to which you trust the output of a language model is on you. But in fairness, hiring the average dev from Fiverr gives you the same experience but a few orders of magnitude slower.

Leave a Comment

Alexander Arvidsson sets expectations:

You’ve written a prompt. It works beautifully. You ship it to production.

Three days later, someone reports wildly different answers to identical questions. You run the exact same input and get a different result than yesterday. Your test suite passes locally, fails in CI, passes again on re-run.

Welcome back to non-determinism in Large Language Models.

Click through for some practical tips on how you can reduce non-deterministic behavior, as well as the trade-offs of doing so.

Leave a Comment

Brent Ozar shares some thoughts:

In the PollGab question queue for Office Hours, MyRobotOverlordAsks asked a question that merited a full blog post answer:

My company announced during some AI training that within the next 12 months we won’t be writing any of our own code. Instead, we’ll be babysitting agents. What’s your opinion on this from a DB dev / DBA POV? MSSQL Dev tends to lag, so I’d personally be surprised.

If this sounds completely alien to you, check out this blog post by developer Armin Ronacher. In it, he discusses how 2025 was the year when he reluctantly shifted his development process to the point where now he spends most of his time doing exactly what MyRobotOverlordAsks’ company is proposing: rather than writing the code directly, he now asks AI tools to build and debug things for him, and he spends his time tweaking what they produce. (Update 2025/01/07: for another example, check out Eugene Meidinger’s post on his uses of AI.)

Brent is generally bullish on the idea. I agree that a lot of companies will move in this direction, but am not at all bullish that it’ll work well. I think this is mostly the latest iteration of Stack Overflow-driven development, except with less copy and paste of bad code and more generation of bad code.

If you want the really spicy version of this take, you’ll have to talk to me in person.

Comments closed

Tomaz Kastrun continues a series on SQL Server 2025 with several posts on vector data. First up is the new vector data type:

The vector data type is designed to store vector data optimized for operations such as similarity search and machine learning applications. Vectors are stored in an optimized binary format but are exposed as JSON arrays for convenience.

Implicit and explicit conversion from and to the vector type can be done using varchar, nvarchar, and json types.

Second is information on vector functions:

Yesterday we looked into Vector data type and how to create table, insert the vector and read it. With SQL Server 2025, vector data type comes equipped also with couple of functions:

And third is how to generate embeddings and store the results in SQL Server:

AI_GENERATE_EMBEDDINGS is a built-in function that creates embeddings (vector arrays) using a pre-created AI model definition stored in the database.

Before running, we need to register the model; creating the master key, database scope credentials and Creating external model.

Comments closed

John Mount describes a problem:

I would like to write a bit about text. That is: technical writing, legal briefs, or even an opinion piece such as this note. Such writings make up much of our society and form a “marketplace of ideas.”

Texts are now very cheap to produce using large language models (LLMs). Some simulated texts remain correct and useful, and some contain numerous subtle flaws and fabrications. In my opinion it remains expensive to reliably determine which text is which type, as LLMs are not as good at detection as fabrication.

Read on for some of the challenges that have come with the proliferation of language models and text auto-generation. John mentions scientific conferences being overwhelmed with AI-generated abstracts, peer reviews, and the like. In the technical world, we’re also seeing an inundation of AI-generated abstracts. For example, we’ve developed a few key tells for submissions to speak at our user group and will automatically reject abstracts that hit those tells. I’m sure there’s a false positive rate there, but that kind of protection mechanism is important to avoid no-shows from artificially generated abstracts.

Comments closed