Press "Enter" to skip to content

Curated SQL Posts

(Re-)Design For Today’s Needs

Published 2016-10-12 by Kevin Feasel

Andy Levy sees common problems when dealing with brownfield applications:

The primary system I deal with on a daily basis was originally developed as a DOS application and several of the above examples are drawn from it. Looking at the core tables and columns, it’s easy to identify those that began life in those early days – they all have 8-character names. Time moved on and the system grew and evolved. DOS to Windows. Windows to the web. But the database, and the practices and patterns used in the database, haven’t come along for the ride.

Data schema conversions can be hard and disruptive – you need to update your application, your stored procedures, and provide customers/users with a clean migration path. Code changes require testing. Complexity and cost grows every time you introduce changes. I get that.

There’s a lot of effort in Andy’s advice, but it’s well worth it.

Comments closed

Test Restores

Published 2016-10-12 by Kevin Feasel

Steve Jones implores you to test those database backups by restoring them somewhere:

What do you do? Hopefully you recognize the issue and can fix the issue. Maybe more importantly, you have a backup of the missing certificate.

Most people don’t deal with encryption, but you never know when your backup job might start failing, perhaps writing to a damaged file that appears to work (if you write as a device) but really isn’t capturing the backup file. Perhaps you don’t know that your backups are being written to a location and deleted a day later, but the process that is supposed to copy them to tape or a remote file share is broken.

Any number of things can happen. The point is that you want to be sure that you are actually getting useable backup files.

That means testing restores.

Read the whole thing.

Comments closed

CAP_CPU_PERCENT

Published 2016-10-12 by Kevin Feasel

Robert Davis looks at the CAP_CPU_PERCENT option in Resource Governor:

The need for this setting came about because MAX_CPU_PERCENT is not applied unless the server is busy. This could lead to a situation where queries in a low priority resource pool starts running while the server is idle and are allowed to consume all the CPU they can. Then high priority queries spin up, and they can’t immediately get the CPU they need due to the low priority queries not being capped. CAP_CPU_PERCENT came along and was designed to set a hard limit that the queries in a pool could not go over even if the server is idle. For example, if you cap the CPU at 25%, the queries in the pool will not exceed 25% no matter how idle the server is.

Problem solved, right?

When the end of a section is a yes/no question, the answer is usually “no.”  Read on before this burns you.

Comments closed

Nothing New Under The Sun

Published 2016-10-12 by Kevin Feasel

Kevin Hill reminisces and warns:

Installation defaults that are going to bite you (not version specific, and the installer is getting better):

  • Files all on the C drive

  • One TempDB data file (improved in SQL 2016)

  • Backups on C drive

  • No automated backups

  • Allow SQL to use ALL the memory

  • Allow SQL to use ALL the CPUs

  • Builtin\Administrators group not default*

  • Compressed backup set to OFF

There’s good advice here, so read on.

Comments closed

Who Monitors The Monitors?

Published 2016-10-12 by Kevin Feasel

Dave Mason discusses monitors and what happens when they fail:

I was reminded of this recently in my little SQL Server world. I have a number of garden variety alerts set up, plus some other more custom monitoring stuff, which is mostly tied to DDL triggers and event notifications. The one thing all of them have in common is database mail. You can probably guess where I’m going with this. Yep, database mail stopped working. A couple weeks passed before I realized it. Fortunately, out of all the alerts I should have been notified about, none of them were serious.

How would I prevent this happening in the future? I guess I could build another system to monitor my monitoring system. Something like System C, which monitors System B, which monitors System A. But where would that end? System D? System E? Where should the line be drawn? I don’t know that there’s a right answer here, although admittedly, the farther into the alphabet you get, the more absurd it sounds.

At some level, process becomes the answer.  In my case, not before I create a few more systems…

Comments closed

KB2919355

Published 2016-10-12 by Kevin Feasel

Allan Hirt notes that you need KB2919355 installed before you can install SQL Server 2016 on Windows Server 2012 R2 or Windows 8.1:

In my case, I created a new VM with a fresh installation of Windows Server 2012 R2. I also ran Windows Update to ensure it had everything Windows Server thought it required. Figure 3 reflects this status.

As you can see in Figure 4, KB2919355 is not listed as one of the ones WU installed, so it has to be an optional update.

Looking at the list of optional updates in Windows Update in Figure 5, 2919355 is not shown. This means you need to download and install it manually.

It’s never quite as easy as “just run this patch,” so do read on for another gotcha.

Comments closed

Kafka Consumer Group Assignment

Published 2016-10-11 by Kevin Feasel

David Brinegar discusses how consumers within an Apache Kafka consumer group get assigned work:

Or one might want some assignment that results in uniform workloads, based on the number of messages in each partition.  But until we have pluggable assignment functions, the reference implementation has a straightforward assignment strategy called Range Assignment.  There is also a newer Round Robin assignor which is useful for applications like Mirror Maker, but most applications just use the default assignment algorithm.

The Range Assignor tries to land on a uniform distribution of partitions, at least within each topic, while at the same time avoiding the need to coordinate and bargain between nodes.  This last goal, independent assignment, is done by each node executing a fixed algorithm:  sort the partitions, sort the consumers, then for each topic take same-sized ranges of partitions for each consumer.  Where the sizes cannot be the same, the consumers at the beginning of the sorted list will end up with one extra partition.  With this algorithm, each application node can see the entire layout by itself, and from there take up the right assignments.

Click through to see an example of how this is implemented.

Comments closed

Dockerizing ASP.Net Applications

Published 2016-10-11 by Kevin Feasel

Elton Stoneman shows how to package an ASP.Net web application into a Docker image:

It’s how you start to package “legacy” ASP.NET apps in Docker images, so you can run them in containers on Windows 10 and Windows Server 2016. Once you’ve packaged your app into a container image you have:

  • a central artifact which dev and ops teams can work with, which helps you transition to DevOps;

  • an app that runs the same on your laptop, on the server, on Azure, on AWS, which helps you move to the cloud;

  • an app platform which supports distributed systems, which helps you break down the monolith into microservices.

This is part one of a series, but if you read through this post, you’ll end up with a fully-functional app.

Comments closed

Scheduler Timing

Published 2016-10-11 by Kevin Feasel

Ewald Cress continues his look at schedulers:

To simplify things initially, we’ll forget about hidden schedulers and assume hard CPU affinity. That gives us an execution environment that looks like this:

  • Each CPU is physically tied to a scheduler.

  • Therefore, out of all the workers in the system, there is a subset of workers that will only run on that CPU.

  • Workers occasionally hand over control of their CPU to a different worker in their scheduler.

  • At any given moment, each CPU is expected to be running a worker that does something of interest to the middle or upper layers of SQL Server.

  • Some of this useful work will be done on behalf of the worker’s scheduler siblings.

  • However, a (hopefully) tiny percentage of a worker’s time is spent within the act of scheduling.

As usual, this is worth the read.

Comments closed

What Is Kafka?

Published 2016-10-11 by Kevin Feasel

I start a new series on Apache Kafka:

The broker serves several purposes:

  1. Know who the producers are and who the consumers are.  This way, the producers don’t care who exactly consumes a message and aren’t responsible for the message after they hand it off.
  2. Buffer for performance.  If the consumers are a little slow at the moment but don’t usually get overwhelmed, that’s okay—messages can sit with the broker until the consumer is ready to fetch.
  3. Let us scale out more easily.  Need to add more producers?  That’s fine—tell the broker who they are.  Need to add consumers?  Same thing.
  4. What about when a consumer goes down?  That’s the same as problem #2:  hold their messages until they’re ready again.

So brokers add a bit of complexity, but they solve some important problems.  The nice part about a broker is that it doesn’t need to know anything about the messages, only who is supposed to receive it.

This is an introduction to the product and part one of an eight-part series.

Comments closed