Press "Enter" to skip to content

Curated SQL Posts

Moving Queries to and from Power BI Desktop and Excel

Published 2020-04-24 by Kevin Feasel

Erik Svensen shows how we can copy queries from Power BI’s Power Query Editor into Excel and back:

Today I discovered a way to easily copy all queries (including the groups) from a PBIX file to Excel without entering the power query editor in Excel.

In my case I needed to copy the data returned by a query and find it some much easier to get the data using Excel instead of generating a csv file from a visual in Power BI Desktop and importing that into Excel.

Click through for a step-by-step demonstration of the process.

Comments closed

Unioning Datasets in Azure Data Factory

Published 2020-04-24 by Kevin Feasel

Rayis Imayev takes us through the Union Transformation in Mapping Data Flows:

Recently I had a chance to work on a special case of creating a file where the first two rows were static and contained output table metadata (columns names and column types), assuming that further reading process would be able to explicitly identify each column data type and consume new file correctly in a single run.

Read on to see how Rayis solved this with Azure Data Factory.

Comments closed

Filtering by Distance in Cosmos DB

Published 2020-04-23 by Kevin Feasel

Hasan Savran shows how we can write queries against spatial data in Cosmos DB:

We are ready to use spatial functions to retrieve data. I have the location of Hurricane Katrina. I want to find my customers who might be affected by Katrina. I want to use the Distance function for this. I have the locations of eye of Katrina in my database, first I will pick one of those locations then look for my customers close to that location. How close? You can control the distance with spatial function named distance, In the following example, I use 100 km which is approximately 62 miles.

Hasan has a demo for us, so check it out.

Comments closed

Currying in Scala

Published 2020-04-23 by Kevin Feasel

Sarfaraz Hussain takes us through one of the most important concepts in functional languages:

Normally we write function and it seems like below:

def multiplySimple(a: Int, b: Int): Int = a * b

We declare a function with all the arguments needed inside a single parameter list.

In currying however, we can split this parameter list into multiple parameter lists.

def multiplyCurry(a: Int)(b: Int): Int = a * b

This doesn’t seem like much, but it does lead to partial application of functions and is one method for how we can have multiple inputs in a functional programming language when mathematical functions allow one input.

Comments closed

Running SQL Server on a Raspberry Pi

Published 2020-04-23 by Kevin Feasel

Andrew Pruski walks us through Azure SQL Database Edge:

This allows SQL Server to run on ARM devices which will expand the range of SQL Server considerably.

Just think how many devices are out there that run ARM. That includes my favourite device, the Raspberry Pi.

So, let’s run through how to get SQL running on a Raspberry Pi!

The process is fairly simple—among all of the steps, the one I had the most trouble with was actually finding the right marketplace, as I ended up in a different one somehow.

Comments closed

Dynamic String Formats for Power BI Calculation Groups

Published 2020-04-23 by Kevin Feasel

Kasper de Jonge shows how we can build out different format strings based on calculation groups in Power BI:

One of the cool things of calculation groups is they cannot just be used to apply a calculation over your “base measure” but also apply a dynamic formatstring. You could do this without calculation groups before too as I described here. This method had one big drawback though, it uses the FORMAT function and when doing that all results are transformed into strings. This does help in showing the right format, but you lose a lot of other functionality like sorting or conditional access. Now with calculation groups we can do custom formatting and keep the data type. How does that work?

Read on for an example.

Comments closed

Using Plan Guides to Bootstrap Query Store

Published 2020-04-23 by Kevin Feasel

Hugo Kornelis gives us a way to use Query Store to force plans on a different server, using plan guides as the instrument:

Query Store only allows you to force plans that the Query Store has “seen” on that instance, and in that database. If you have a query and you want to force a specific plan, you will need to first ensure that the query runs, at least once, under the right circumstances to create the desired plan, so that the Query Store can capture it. Sometimes that is easy, in which case this blog is not for you.

What if you know you need to force the execution plan, you know you’ll get that execution plan only when a lot of data is already in the database, and you need to ship your software with an empty database to your customers? How do you set it up to force the plan you need?

Click through for the answer, and I give Hugo bonus points for using Raleigh as the example.

Comments closed

Java Extension for SQL Server Now Open Source

Published 2020-04-23 by Kevin Feasel

Nellie Gustafsson announces some exciting news:

Today, we’re thrilled to announce that we are open sourcing the Java language extension for SQL Server on GitHub.

This extension is the first example of using an evolved programming language extensibility architecture which allows integration with a new type of language extensions. This new architecture gives customers the freedom to bring their own runtime and execute programs using that runtime in SQL Server while leveraging the existing security and governance that the SQL Server programming language extensibility architecture provides.

This opens up the possibility for additional languages. .NET languages (C# and F#) would be a natural fit and languages like Go might have enough dedicated support to give this a try.

Comments closed

Date and Time Storage in SQL Server

Published 2020-04-23 by Kevin Feasel

Randolph West covers the internals of how date and time data types are stored in SQL Server:

DATE is the byte-reversed number of days since the year 0001-01-01, stored as three bytes. It goes up to 9999-12-31, which is stored as 0xDAB937. You can check this value by reversing the bytes and sticking them into a hex calculator. 37 B9 DA equals 3,652,058, which is the number of days since 0001-01-01.

If you try to cast 0xDBB937 as a DATE value (by incrementing the least significant bit DA by 1), it will throw a conversion error. There is obviously some overflow detection that protects against corruption in a date type.

Randolph looks at DATE, TIME, DATETIME(2), and DATETIME and explains how each is storedon a page.

Comments closed

Automated ML and Data Scientists

Published 2020-04-22 by Kevin Feasel

Sophia Rowland takes us through an experiment:

Ever since automated machine learning has entered the scene, people are asking, “Will automated machine learning replace data scientists?” I personally don’t think we need to be worried about losing our jobs any time soon. Automated machine learning is great at efficiently trying a lot of different options and can save a data scientist hours of work. The caveat is that automated machine learning cannot replace all tasks. Automated machine learning does not understand context as well as a human being. This means that it may not know to create specific features that are the norm for certain tasks. Also, automated machine learning may not know when it has created things that are irrelevant or unhelpful.

To strengthen my points, I created a competition between myself and two SAS Viya tools for automated machine learning. To show that we are really better together, I combined my work with automated machine learning and compared all approaches. Before we dive into the results, let’s discuss the task.

The results are in line with my expectations: a good automated ML tool will make life easier, but doesn’t replace the expert system of a human.

Comments closed