Press "Enter" to skip to content

Curated SQL Posts

Spark Access Control in Qubole

Published 2019-07-29 by Kevin Feasel

Achuth Rajagopal and Shridhar Ramachandran show off the Spark Data Access Control Framework on Qubole’s platform:

With these requirements in mind, we decided to implement Hive Authorization as our first Policy Manager. Hive Authorization policies are stored in the Qubole Metastore which acts as a shared central component and stores metadata related to Hive Resources like Hive Tables. We enhanced Spark to honor the policies stored in the Qubole Metastore while accessing Hive Tables or for adding and modifying those policies.

In summary, we implemented a SQL standard access control layer identical to what is present in Apache Hive or Presto today. The following sections detail the architecture and provide an example that illustrates how it works.

Click through to learn more.

Comments closed

Data Virtualization and Power BI

Published 2019-07-29 by Kevin Feasel

Gerhard Brueckl explains how Power BI can act as a data virtualization engine:

But lets examine what we currently have in Power BI:
– a semantic layer and data modelling capabilities
– access to various data sources via Direct Query (remember, we do not want to load any data!)
– ability to combine data from those sources

The last part is the most important one here and you may wonder what I am talking about. And you are right, by default a DQ model is only linked to one data source at a time but you can add other data sources manually in the Power Query editor! 

This is where it get’s interesting and what this blog post is about.

Read the whole thing.

Comments closed

Filtering in DAX

Published 2019-07-29 by Kevin Feasel

Matt Allington takes us through the FILTER() function in DAX:

FILTER() is most often used as a filter parameter within the calculate function. OK, so now it is getting confusing. FILTER() is a function. There is another use of the word filter in DAX, and that is as a parameter in CALCULATE. Let’s look at the syntax

FILTER Syntax: FILTER(<table>, <true/false test>) CALCULATE Syntax: CALCULATE (<measure expression>, <filter parameter>, <filter parameter>,…)

This is one of the most useful functions in DAX and one you really want to know well.

Comments closed

Data Aggregation with Powershell

Published 2019-07-29 by Kevin Feasel

Jess Pomfret shows us how we can aggregate data using Powershell:

As a SQL Server DBA, aggregating data is first nature.  I can easily throw together some T-SQL if I need to get the average sales per product group, or perhaps the number of employees hired per month. Yesterday I was writing a little PowerShell when I came across a problem. I needed to get the size of my database data and log files for several databases on a server, when I realized I didn’t know how to group and sum this data in PowerShell.

Click through to learn how.

Comments closed

Undocumented Commands Can Change

Published 2019-07-29 by Kevin Feasel

Thomas Rushton warns us that undocumented commands in SQL Server are liable to change without notice:

I don’t have every version available to test, but it does appear that the record indicating Containment State is mis-spelled in SQL Server 2012 as “dbi_ContianmentState”, and then corrected in SQL 2014 and later. It’s a good job I’m not relying on it for anything.

So, yes, don’t rely on undocumented functions – as they may change without notice.

If there’s a documented method, use that one. If the only method available is undocumented, you can still use it, but be sure to test it with each release—that is, major release, service pack, or cumulative update.

Comments closed

Using the Power BI Visual Header Tooltip

Published 2019-07-29 by Kevin Feasel

Prathy Kamasani gives us several good uses of the Power BI visual header tooltip:

When we look at data journalism posts, most of the times they have annotations, explaining what visual showing or talking about measures. Again most of these data storeys are used for paper. But in the digital world, we do see these annotations more interactively. It is nice to have this kind of lil annotations for everyday reporting as well, and Tooltip Icon can be used for that purpose. Another thing is using canvas space wisely, it is important, and having this kind of hint helps us on saving the canvas space.

Click through for instructions on how to enable this as well as smart ways to use them.

Comments closed

Automated Alert Emails

Published 2019-07-29 by Kevin Feasel

Max Vernon shows how you can use the SQL Server Agent to send automated e-mails on alerts:

SQL Server Agent provides a great mechanism for sending alerts via email in response to various events. Once you’ve setup Database Mail, and configured a SQL Server Agent Operator, you should add alerts for severe errors that affect the health of your SQL Server. Creating Alerts can be tedious, but automating Alerts is simple, with the easy code below that automates creating alerts in response to critical events. Automating alerts is important because it provides a standardized Alert configuration that can be used by all the SQL Servers in your organization.

Read on for the script.

Comments closed

SSMS 18.2 Available

Published 2019-07-26 by Kevin Feasel

Dinakar Nethi announces SQL Server Management Studio version 18.2:

We are excited to announce the release of SQL Server Management Studio (SSMS) 18.2. For this update, while we added some features, our focus was dedicated to fundamentals such as stability, reliability, performance, etc.

You can download SQL Server Management Studio 18.2 today.
Some of the new features in SQL Server Management Studio include:

– Intellisense/editor: Added support for data classification
– Query execution: Added a completion time in the messages to track when a given query completed its execution.
– ShowPlan: Added new attribute in query plan when the inline scalar UDF feature is enabled.

There are several bugfixes in there as well.

Comments closed

A Primer on Jenkins

Published 2019-07-26 by Kevin Feasel

Shubham Dangare gives us a quick walkthrough of setting up a Jenkins pipeline:

Jenkins Pipeline (or simply “Pipeline”) is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins.

A continuous delivery pipeline is an automated expression of your process for getting the software from version control right through to your users and customers.

It provides an extensible set of tools for modeling simple-to-complex delivery pipelines “as code”. The definition of a Pipeline is typically written into a text file (called a Jenkinsfile ) which in turn is checked into a project’s source control repository

Click through for a demo.

Comments closed

Defining Intent Locks

Published 2019-07-26 by Kevin Feasel

David Fowler explains what an intent lock is and why it’s useful:

Let’s just imagine a World without intent locks for a moment. In that World, a user has just decided to select a row from our database. SQL at that point is going to put down a shared lock against the row.

Now what’s going to happen when another user decides to modify a bunch of rows? Now because of the number or rows involved in this modification, SQL is going to want to take out an exclusive page lock. Where’s the issue here?

Read on to learn what the issue is.

Comments closed