Press "Enter" to skip to content

Author: Kevin Feasel

The Luminance Illusion With gganimate

Published 2018-08-23 by Kevin Feasel

David Smith highlights an example of the luminance illusion:

Colin created this animation in R using the gganimate package (available on GitHub from author Thomas Lin Pederson), and the process is delightfully simple. It begins with a chart of 10 “points”, each being the same grey square equally spaced across the shaded background. Then, a simple command animates the transitions from one point to the next, and interpolates between them smoothly:

library(gganimate)
gg_animated <- gg + transition_time(t) + ease_aes('linear')

Check it out, both as a parlor trick and a way of getting a grip on the gganimate package.

Comments closed

Styling In ggplot2

Published 2018-08-23 by Kevin Feasel

The folks at Jumping Rivers show an example of creating a nice-looking plot with ggplot2:

The changes we’ve made so far would impossible for any package to do for us – how would the package know the plot title? We can now improve the look and feel of the plot. There are two ways of complementary ways of doing this: scales and themes. The ggplot scales control things like colours and point size. In the latest version of ggplot2, version 3.0.0, the Viridis colour palette was introduced. This palette is particularly useful for creating colour-blind friendly palettes

g + scale_colour_viridis_d() # d for discrete

With a few lines of code, those default graphs can look a lot nicer.

Comments closed

Formatting Queries As JSON With FOR JSON

Published 2018-08-23 by Kevin Feasel

Eduardo Pivaral shows off the FOR JSON functionality in SQL Server 2016 and later:

For most of real-world applications, the JSON AUTO will not give you the control you could need over your file format, for having more control over it, you must use the JSON PATH option, along with the ROOT option as follows:

SELECT TOP 10 id, dataVarchar, dataNumeric, dataInt, dataDate
FROM [dbo].[MyTestTable]
FOR JSON PATH, ROOT('TestTable')

Eduardo has several examples along these lines.

Comments closed

KPIs In A Table With Power BI

Published 2018-08-23 by Kevin Feasel

Kasper de Jonge shows us how to embed SVGs into a Power BI table or matrix to show visual forms of KPIs:

Got a question today on how we can do KPI’s in the matrix or table with Power BI, just like we can with PowerPivot through the KPI functionality there.

This is where the great SVG trick from David Eldersveld comes into play, you can read all about it here in his blog posts: https://dataveld.com/2018/01/13/use-svg-images-in-power-bi-part-1 and this post by Chris Webb with even more cool examples.

In this case I am using his trick and a post I found on the community site to add a KPI to my table.

Read on for an example.

Comments closed

Azure SQL Database Elastic Job False Successes

Published 2018-08-23 by Kevin Feasel

Arun Sirpal walks us through a case where an Azure SQL Database elastic job appears to be running correctly but isn’t:

I use elastic pools. They are a fabulous way of saving money when running many Azure SQL Databases, that is assuming you understand the resource utilization patterns of the databases involved.

I recently created couple of elastic jobs for my pool and went through a successful setup (so I thought) and proceeded to execute the query. The query being elastic in nature basically runs across all my databases from a single point. This is where it got puzzling, if you look at the below screen shot you can clearly see that the status is RUNNING, drilling into it, it says RETRYING. This obviously is not right.

Click through to see the root cause of this problem.

Comments closed

Causing tempdb Spills

Published 2018-08-23 by Kevin Feasel

Kendra Little shows us a quick and easy way to cause tempdb spills:

Sometimes it’s useful to know how to cause a problem.

Maybe you’ve never encountered the problem, and want to get hands-on experience. Maybe you’re testing a monitoring tool, and want to see if a condition flags an alert. Maybe you’re testing out a new client tool, and want to see how it displays it.

I recently was going through some demos in SQL Operations Studio, and I found that a spill on a sort operator wasn’t causing a warning to visibly show in the graphic execution plan.

I wanted to file an issue on this and let the Ops Studio team know that would be helpful – but my demo code was somewhat complex and required restoring a rather large database. So I set up a quick code sample to cause a spill that could be run in any database.

It’s important to know how to cause problems if you want to make sure you’ve solved them correctly.

Comments closed

Ambari Architecture

Published 2018-08-22 by Kevin Feasel

The folks at Data Flair have a tutorial on how Ambari is architected:

Ambari Architecture is of master/slave type architecture. So, to perform certain actions and report back the state of every action, the master node instructs the slave nodes. Although, for keeping track of the state of the infrastructure, the master node is responsible. But for this process, a database server is used by the master node, that can be further configured during setup time.

Now, we can see the high-level architecture of Ambari by below diagram which also shows how Ambari works:

Ambari is one of the easiest ways I’ve seen to spin up and manage a Hadoop cluster.

Comments closed

Matrix Math In R

Published 2018-08-22 by Kevin Feasel

Dave Mason continues his series on matrices in R:

Math operations between matrices is possible too. Here, the same matrix is added to itself. Since it’s the same matrix, they obviously have the same number of elements. The first element is added to the first element, the second element is added to the second element, etc.

> #Add two matrices.
> some_numbers + some_numbers [,1] [,2] [,3] [,4] [,5] [,6]
[1,] 2 4 6 8 10 12
[2,] 14 16 18 20 22 24
[3,] 26 28 30 32 34 36
[4,] 38 40 42 44 46 48

This follows from Dave’s prior posts, but you can see some of the pieces start to fit together.

Comments closed

Kerberos And SQL Server

Published 2018-08-22 by Kevin Feasel

Kathi Kellenberger digs into Kerberos:

2. Why is Kerberos needed for SQL Server?

When NTLM is used, the client, for example a user logged into a laptop, contacts a domain controller when requesting access to a resource in the network. This resource could be an SSRS report, for example. When using NTLM, the user proves their identity to the SSRS server. Unfortunately, the SSRS server cannot forward the credentials of the user along to the database server. The database server will deny the request, and the end user will see an error message. This is common with SSRS but will also be seen whenever resources are needed involving multiple servers.

When Kerberos is property configured, the SSRS server can pass along confirmation of the identity of the requester to the database server via the ticket. If the login of the original requester has permission to select the data, it’s returned to the SSRS server, and the report is delivered.

Even if you are not using SSRS, you can run into issues when Kerberos is not configured properly. For example, you will often see error messages when trying to connect to SQL Server using SSMS (SQL Server Management Studio) when logged into another server when SPNs are misconfigured.

Having a good understanding of Kerberos can save you configuration headaches when going between servers.

Comments closed

Moving Data Between Data Lakes

Published 2018-08-22 by Kevin Feasel

Jeffrey Verheul shows us how to use AdlCopy to migrate data from one Azure Data Lake to another:

Migrating data from one Data Lake to the other
We started out with a test version of a Data Lake, and this week I needed to migrate data to the production version of our Data Lake. After a lot of trial and error I couldn’t find a good way to migrate data. In the end I found a tool called AdlCopy. This is a command-line tool that copies files for you. Let me show you how easy it is.

Download & Install
AdlCopy needs to be installed on your machine. You can find the download here. By default the tool will install the files in “C:\Users\\Documents\AdlCopy\”, but this can be changed in the setup wizard.

Click through to see how to use this tool.

Comments closed