Press "Enter" to skip to content

Curated SQL Posts

Accessing the Purview Portal in Your Fabric Environment

Published 2024-02-12 by Kevin Feasel

Kevin Chant enables a feature:

In this post I want to cover accessing the new Microsoft Purview portal in your own Microsoft Fabric environment.

To clarify, I mean a Microsoft Fabric environment you have created for your own use. Like the one I covered in a previous post.

You can do this in a trial environment thanks to the new capability provided by Microsoft last year to infuse Microsoft Fabric items into Microsoft Purview. Which Microsoft covered in a blog post about Microsoft Fabric items in Microsoft Purview.

Read on to see how.

Comments closed

Reading Parquet Metadata in Power Query

Published 2024-02-12 by Kevin Feasel

Chris Webb taps into the source:

There’s a new M function in Power Query in Power BI that allows you to read the data from a Parquet file: Parquet.Metadata. It’s not documented yet and it’s currently marked as “intended for internal use only” but I’ve been told I can blog about it. Here’s an example of how to use it:

Click through to see what you can get in a Parquet file. This is part of the secret of why the Parquet file format is so fast.

Comments closed

Automating Microsoft Fabric Capacity Scaling via Logic App

Published 2024-02-12 by Kevin Feasel

Soheil Bakhshi does some scaling:

In a previous post I explained how to manage the capacity costs of a Fabric F capacity (under Pay-As-You-Go pricing model) using Logic Apps to Suspend and Resume it.

A customer who read my previous blog asked me “Can we use a similar method to scale up and down before and after specific workloads?”. This blog post is to answer exactly that.

This is pretty neat, though I wonder how long it takes and how much downtime it produces.

Comments closed

Building Functions with Spark Connect and .NET

Published 2024-02-09 by Kevin Feasel

Ed Elliott continues a series on Spark Connect:

I’m pretty much going to leave the code as-is from the previous post but will move things about a bit and add a SparkSession and a DataFrame class. Also, instead of passing the session id and client around i’m going to wrap them in the SparkSession so that we can just pass a single object and also use it to construct the DataFrame so we don’t even have to worry about passing it around.

The first thing is to take all of that gRPC connection stuff and shove in into SparkSession so it is hidden from the callers:

Read on for the end state that Ed is headed toward and how to get closer to that state.

Comments closed

Recompile Hints and Query Store

Published 2024-02-09 by Kevin Feasel

Erik Darling talks about one very popular tool and one very popular technique and what happens when they mix:

But let’s say one day you have performance issues regardless of all the recompiling in the world. No matter what set of parameters get passed in, your procedure is just sleepy-dog slow.

Things get so bad that you hire a young, handsome consultant with reasonable rates like myself to help you figure out why.

The plan cache will be pretty useless for troubleshooting the recompile-hinted queries, but we can still use Query Store.

Read on for more detail and a demonstration.

Comments closed

Oracle SQL Developer for Visual Studio Code

Published 2024-02-09 by Kevin Feasel

Brendan Tierney takes a look at a new extension for VS Code:

We now have a new/different tool for developers to access their Oracle Databases. Traditionally, developers have been using Oracle SQL Developer for maybe 20+ years (if you started using Project Raptor). SQL Developer has developed into a bit of a big beast of a tool, with it trying to be everything to everyone including developers, DBAs, and others. But it does seem like SQL Developer might be coming to an end of life, although that could be for some years to come as it is so wildly used. There have been many challenges with SQL Developer over the years and one of the main challenges is getting new developers to use it. From my experience, developers tell me they just didn’t like it, didn’t like the look and feel of it, it was difficult to use, etc., etc. The list would go on and on and most of those developers would prefer to use other tools (for example DBeaver). For those that are terminal/command line only person, you have SQL*Plus and the modern version called SQLCl (SQL Command Line).

Read on for a review of how the extension is today, with the promise of regular improvements from Oracle over time.

Comments closed

Finding the Week Number in R

Published 2024-02-08 by Kevin Feasel

Steven Sanderson checks the week:

When working with dates in R, you may need to extract the week number for any given date. This can be useful for doing time series analysis or visualizations by week.

In this post, I’ll demonstrate how to get the week number from dates in R using both base R and the lubridate package. I’ll provide simple examples so you can try it yourself.

Steven also makes a good point about ISO weeks (which are common in Europe) versus calendar weeks.

Comments closed