Press "Enter" to skip to content

Curated SQL Posts

Streaming Data With Kinesis

Asaaf Mentzer shows how to join streaming data (specifically, AWS Kinesis) with lookup data:

In this use case, Amazon Kinesis Analytics can be used to define a reference data input on S3, and use S3 for enriching a streaming data source.

For example, bike share systems around the world can publish data files about available bikes and docks, at each station, in real time.  On bike-share system data feeds that follow the General Bikeshare Feed Specification (GBFS), there is a reference dataset that contains a static list of all stations, their capacities, and locations.

There are three different architectures in here, so if you’re looking for streaming data models with Kinesis (or want to apply them to Kafka), this is a solid read.

Comments closed

Dial Gauge

Devin Knight explains the dial gauge custom visual:

  • The effectiveness of gauges on dashboards is an often debated topic.

  • The Dial Gauge is completely data driven. Which means not only must your measure (drives the needle) come from a dataset but also the different thresholds ranges must come from your dataset too.

  • There are no specific Format settings for the Dial Gauge, which does limit you a bit with what you can do with this gauge.

There are certain scenarios in which I think the dial gauge works well.  The best scenario is the the same as its analog counterpart:  when you are measuring a single continuous variable with a safe range and meaningful range differences.  This scenario occurs less often than you might think.

Comments closed

Cannot Connect To WMI Provider

Andrew Peterson troubleshoots an error after installing SSMS vNext:

After installing SQL Server Management Studio for vNext, the Configuration Manager no longer opens, with a message similar to the following:

Cannot connect to WMI provider. You do not have permission or the server is unreachable. Note that you can only manage SQL Server 2005 and later servers with SQL Server Configuration Manager.
Invalid namespace [0x8004100e]

Read on for the solution.

Comments closed

Backup Basics

Aaron Bertrand covers reasons for backups, backup models, and also a vital part of the backup process:

Now, all of the above may be review for you, but a much more important part of this story is that you need to be TESTING your backups. I’ve seen many customers who have been happily taking backups and storing them on some drive somewhere, and then when disaster strikes and they actually need to restore them, they can’t – maybe they had been backing up corruption all along, or the backups were failing but they were ignoring alerts, or they weren’t taking log backups frequently enough to meet their RPO, or they were only taking full backups.

Testing backups is vital; just because the backup process reported success doesn’t mean that you’ll necessarily be able to restore that backup when the time comes that you need it.  It’s also good to drill people on restoration skills, as things get a bit more stressful when three levels of management are standing behind your chair asking you what’s taking so long.

Comments closed

Backing Up Extended Event Logs

Wayne Sheffield reminds us that backups aren’t just for databases:

So how does this talk of AGs pertain to this T-SQL Tuesday topic? It should be pretty obvious – we need to periodically grab all of the .xel files generated by the cluster, and move them to a different directory, with a different retention policy. Yup… we need to back up these files. Sometimes, we need to be backing up things other than the databases themselves.

I created a PowerShell script that takes a few parameters, then moves the files from the source directory to the destination directory. And then it deletes files from the destination directory that are over x days old.

Wayne goes into more detail, including permissions required to run the script.

Comments closed

Restoring To A Specific Time

Derik Hammer shows one of the most useful features of database restores:

In order to restore point-in-time, you need to restore the full backup with NORECOVERY. This tells SQL Server not to initiate crash recovery which is a process that performs the redo and undo operations on your database to roll back the uncommitted transactions and roll forward the committed ones.

Once the full backup is restored you will need to restore the rest of the LSN (log sequence number) chain in the appropriate order. If you are not using differential backups, this means that you need to restore each log file until you cover the point-in-time that you are targeting. If you do have one or more differential backups, just restore the most recent differential which was taken before your target point-in-time and then all log backups between then and the target.

Derik also discusses restoring to marked transactions, something I’ve never used before but which could be very useful for known, major changes (like database code rollouts).

Comments closed

Bacup With NORECOVERY

Robert Davis explains the NORECOVERY option when running a backup (rather than restore) command:

NORECOVERY applies only to log backups. When you run a log backup with NORECOVERY it takes the normal log backup and it also puts the database into a restoring state. This means that absolutely no transactions can run in the at database that isn’t covered by the log backup. At least not until someone recovered it. So is a scenario coming clear where that might be helpful? Think migrations.

Read on for more details.  It’s not something you’ll probably do frequently, but knowing it may make a future operation smoother.

Comments closed

Upgrading Backups

Steve Jones explains that you can restore most backups to a later version, but can never restore to an earlier major version:

I would hope that most people know that a SQL Server database backup has a version. This version corresponds to a version of SQL Server, and for the most part, we can’t restore a database backup to an earlier version of SQL Server. Some exceptions might be a similar CU version there the database format hasn’t changed, but certainly not to any prior Service Pack.

However, can you restore to a later version? Can I take a SQL Server 2012 database backup and restore it to a SQL Server 2016 instance? Sure I can. In fact, lots of people upgrade their systems this way. Install a new SQL Server instance, take a backup on the old version and bring it forward. In fact, you can restore (or attach) a SQL Server 2005 database backup on SQL Server 2016.

Read on for more details and caveats.

Comments closed

Use Backup Compression

Thomas Rushton advocates for enabling backup compression:

SQL Server backup compression – does what it says on the tin. Instead of SQL Server taking a backup by reading pages / extents of data from the database file and writing them out to the backup file, it compresses the data before it writes. It’s not the best compression you’ll get, as it won’t read the entire file before compressing it; however, it’s good enough that on OLTP databases that contain normal varchar / numeric data you could see savings of over 75% – indeed, I have some databases that save 90% of disk space. And that’s made my storage guys happy. Well, less unhappy.

One side note:  if you’re also encrypting backups (available in the product since SQL Server 2014), encryption happens after compression, so you can take advantage of both.

1 Comment

It’s Not Just Backups

Dave Mason looks at alternatives to restoring databases:

Database Snapshots

A snapshot creates a read-only static view of a source database. With a snapshot, DML statements can be run on the source database and the snapshot database will preserve the original data. The snapshot can be used to “undo” data changes in the source database. There’d likely be more T-SQL/scripting work involved than a simple database restore. However, a snapshot has less “overhead” than a backup (at first). As noted in the MSDN documentation, “As the source database is updated, the database snapshot is updated. Therefore, the longer a database snapshot exists, the more likely it is to use up its available disk space.” In addition, there are prerequisites for and limitations on database snapshots.

Restoring a database backup should be easy, but it might also tell you that there was a failure somewhere.  If you’re regularly restoring backups because of data entry issues, then it might make sense to keep a history of the data so you have tools to fix issues short of the nuclear option.

Comments closed