Press "Enter" to skip to content

Month: November 2017

Columnstore Memory Pressure And Bulk Loading

Published 2017-11-10 by Kevin Feasel

Niko Neugebauer shows what happens when you try to bulk load data into a columnstore index but don’t have enough memory available:

After waiting for the 25 seconds (notice the difference between the request_time and grant_time is exactly 25 seconds), the engine decides to grant some minimum amount of memory anyway, allowing the process to carry on, without being cancelled, but the penalisation is very heavy – the inserts will not go into the compressed row groups, but into the Delta-Stores, making this operation not-minimally-logged and in other words, painfully slow and inefficient.
To confirm the final results, let’s check on the Row Groups of our tables, given that we have canceled the inserts into the 2 first tables, we expect 1 row group for the [dbo].[FactOnlineSales_Stage3] table and 1 row group for the [dbo].[FactOnlineSales_Stage4] table, corresponding to the 3rd and 4th threads of data loading:

As Niko points out, this could be the difference between a well-behaved, single compressed rowgroup load versus dumping a million rows into the deltastore.

Comments closed

SQL Server Single-User Mode In Linux

Published 2017-11-10 by Kevin Feasel

Anthony Nocentino shows us how to get into single-user mode in SQL Server on Linux:

There was a question this morning on the SQL Server Community Slack channel from SvenLowry about how to launch SQL Server on Linux in Single User Mode. Well you’ve heard everyone say, it’s just SQL Server…and that’s certainly true and this is another example of that idea.

The command line parameters from the sqlservr binary are passed through into the SQLPAL managed Win32 SQL Process. So let’s check out how to do this together…

Click through for a demo.

Comments closed

When “Invalid Column Name” Isn’t A Permissions Issue

Published 2017-11-10 by Kevin Feasel

Kenneth Fisher shares a story of hunting down the cause of an error message:

This time we had a vendor reporting the following error:

Msg 207, Level 16, State 1, Line 7
Invalid column name ‘Name’

Now the vendor was certain this was a permissions issue. It worked fine on their systems, it worked fine on some of ours. So why didn’t it always work? Well, the easy answer is permissions! Particularly since we had denied them db_owner just recently.

So why do I sound so dismissive about permissions as a possibility? I mean it COULD be permissions. It certainly is possible. But first of all, we don’t use column level permissions very often (no one uses them all that often from what I can tell) and secondly it worked on several other systems where they had exactly the same permissions as this system.

Ok, so what is the problem? You guessed it! (I really have to stop asking for guesses after I’ve put the answer in the title.)

I went out of my way not to give the answer here, so you’ll have to look at Kenneth’s title.  And then read the whole thing.

Comments closed

Fooling Neural Networks

Published 2017-11-09 by Kevin Feasel

Rodrigo Agundez shows how to fool neural networks:

A comprehensive and complete summary can be found in the When DNNs go wrong blog, which I recommend you to read.

All these amazing studies use state of the art deep learning techniques, which makes them (in my opinion) difficult to reproduce and to answer questions we might have as non-experts in this subject.

My intention in this blog is to bring the main concepts down to earth, to an easily reproducible setting where they are clear and actually visible. In addition, I hope this short blog can provide a better understanding of the limitations of discriminative models in general. The complete code used in this blog post can be found here.

This is a great article.

Comments closed

Grouping Data With ROLLUP

Published 2017-11-09 by Kevin Feasel

Steve Jones shows how easy it is to use the ROLLUP function:

I was editing an article recently that talked about ROLLUP, and I wanted to play with this a bit more. I hadn’t used this much in my career, but it’s a handy operator that’s worth including in your toolkit.

ROLLUP is used to provide additional totals for your aggregates while using GROUP BY. Here’s a little example. I’ve got some data for American Football quaterbacks. In this case, I’ve extracted some stats for a few noteworthy players today.

I’ll probably end up using ROLLUP about once every three months and be very pleased when I remember that it exists.  I use GROUPING SETS more often and almost never use CUBE.

Comments closed

Power BI Query Reduction

Published 2017-11-09 by Kevin Feasel

Wolfgang Strasser points out a new feature in Power BI Desktop:

In some cases, sources for Direct Query models are either large (and therefore need some time to bring back the data) or slow (which could maybe be improved by some database tuning :-)).

Before the Nov2017 release, every change in a filter or slicer led to an immediate round trip to the data source which made some of those reports … let’s call it .. not perfect in the context of user experience.

Beginning with the Nov2017 release some options for Query Reduction has been introduced.

Click through for an example of how to use Query Reduction on direct query models.

Comments closed

TDE As Security Theater

Published 2017-11-09 by Kevin Feasel

Randolph West argues that Transparent Data Encryption is not security theater:

The short version is that our data, log and backup files are encrypted at rest (i.e., on the storage layer), so that an attacker cannot simply copy and attach the data and log files, or restore a backup, without having access to the master key. If backup tapes or drives are stolen, the data on those devices cannot be recovered.

We can also use what is known as a Hardware Security Module (HSM) to provide keys to secure the database. This is a dedicated physical or virtual device, separate from SQL Server, that generates keys for various services in an organization.

Read on for Randolph’s argument.  For the counter-argument, check out Simon McAuliffe’s article on the topic.  Over the past couple of years, I’ve grown much more sympathetic to the idea that TDE’s primary use case is in weeding out the rabble-rousers with nary a clue, but it’s not very helpful against a knowledgeable attacker with administrative access to your SQL Server.

Comments closed

Dynamic Row-Level Security And Power BI

Published 2017-11-09 by Kevin Feasel

Reza Rad shows how to build role-based filters in Power BI and use those to implement row-level security:

 

I have written while ago, about how to implement a dynamic row level security in Power BI. This post is an addition to that post. I’ve had a lot of inquiries that; “What If I want users to see their own data, and the Manager to see everything?”, or “How to add Manager or Director Level access to the dynamic row level security?” This post will answer this question. In this post, you will learn a scenario that you can implement a dynamic row level security. In this scenario, everyone will see their own data, but the manager will see everything. If you want to learn more about Power BI, read Power BI book from Rookie to Rock Star.

Read on to see one method of solving this dilemma.

Comments closed

Automatically Scripting Objects Using dbatools

Published 2017-11-09 by Kevin Feasel

Constantine Kokkinos shows how easy it is to use dbatools to script out database objects:

@brianszadek says: I am looking to automate the scripting of objects with PowerShell for easier source control integration.

I thought this would be a little bit easier, but it looks like the SMO way of doing this is still fairly complicated. Maybe something I can look at simplifying for the project in a bit

If you want to get your database into source control but feel like it’s going to be a time-consuming challenge, this isn’t the time-consuming part.

Comments closed