Press "Enter" to skip to content

Author: Kevin Feasel

Enabling Staging for Microsoft Fabric Dataflows

Published 2023-09-05 by Kevin Feasel

Chris Webb shares some thoughts:

If you read this post that was published on the Fabric blog back in July, you’ll know that each Power Query query in a Fabric Gen2 dataflow has a property that determines whether its output is staged or not – where “staged” means that the output is written to the (soon-to-be hidden) Lakehouse linked to the dataflow, regardless of whether you have set a destination for the query output to be written to. Turning this on or off can have a big impact on your refresh times, making them a lot faster or a lot slower.

Chris shares a simple example of when staging might not be reasonable. This is going to be the less common scenario, however.

Comments closed

The Value of KPIs and Cards in Power BI

Published 2023-09-05 by Kevin Feasel

Kurt Buhler and Stepan Resl give you a card:

When a user arrives at your report, they should be able to answer their most important questions in a few seconds. To do this, we typically put the most critical information in the top-left of the report (where we often look first). This information should provide a high-level overview, whereas additional details should be placed at the bottom of the report, behind interactions, or on later pages.

An effective and popular way to call attention to important numbers in Power BI is by using cards and KPI core visuals.

Read on for several examples and a breakdown of how they work best.

Comments closed

Querying the Power BI REST API from Fabric Spark

Published 2023-09-01 by Kevin Feasel

Gerhard Brueckl makes the call:

Microsoft Fabric has a lot of different components which usually work very well together. However, even though Power BI is a fundamental part of Fabric, there is not really a tight integration between Data Engineering components and Power BI. In this blog post I will show you an easy and reusable way to query the Power BI REST API via Fabric SQL in a very straight forward way. The extracted data can then be stored in the data lake e.g. to create a history of your dataset refreshes, the state of your workspaces or any other information that is provided by the REST API.

Click through for a list of operations, followed by the code you’ll need to pull this off.

Comments closed

Tuning Kafka Connect Source Connectors

Published 2023-09-01 by Kevin Feasel

Catalin Pop makes things faster:

Kafka Connect is an open source data integration tool that simplifies the process of streaming data between Apache Kafka® and other systems. Kafka Connect has two types of connectors: source connectors and sink connectors. Source connectors allow you to read data from various sources and write it to Kafka topics. Sink connectors send data from the topics to another endpoint. This blog post discusses how to tune your source connectors to help you get the best throughput out of your compute resources. 

This includes which elements are tunable, metrics you’ll want to pay attention to along the way, and a detailed example.

Comments closed

String Aggregation with STRING_AGG

Published 2023-09-01 by Kevin Feasel

Jeff Mlakar uses the STRING_AGG() function:

Often when generating dynamic sql we must add some separator values in the string(s) we build. A function delivered way back in 2017 called STRING_AGG is a simple way to do this. Let’s look at some examples.

There are a lot of people who have never heard of the function and don’t know that there’s a better alternative to the combination of STUFF() and FOR XML PATH to concatenate together strings.

Comments closed

PostgreSQL 16 and Infinity

Published 2023-09-01 by Kevin Feasel

Ryan Lambert goes to infinity and beyond:

This month, Ryan Booz chose the topic: What Excites You About PostgreSQL 16? With the release of Postgres 16 expected in the near(ish) future, it’s starting to get real. It won’t be long until casual users are upgrading their Postgres instances. To decide what to write about I headed to the Postgres 16 release notes to scan through the documents. Through all of the items, I picked this item attributed to Vik Fearing.

  • Accept the spelling “+infinity” in datetime input

The rest of this post looks at what this means, and why I think this matters.

Read on to see what’s new about this and what it all means.

Comments closed

STRING_SPLIT Not a Recognized Built-In Function Name

Published 2023-09-01 by Kevin Feasel

Kendra Little resolves an error:

Yesterday, I was writing some Transact SQL to dust off the cobwebs. I got confused when I was playing around with the STRING_SPLIT function, and kept getting the error:

Msg 195, Level 15, State 10, Line 2 ‘STRING_SPLIT’ is not a recognized built-in function name.

When I saw this, I immediately thought compatibility level, as that’s something I’ve run into several times before. In this case, it turns out that the correct answer was something else.

Comments closed

Building a Report Model from Agile User Stories

Published 2023-09-01 by Kevin Feasel

Kelly Broekstra digs into a story:

Are you a model designer or BI developer tasked with building a data model and/or report from a series of user requirements or Agile User Stories? Do you know where to start, or what to try first? Here are some practical tips and techniques that I have used to design a great report model from Agile user stories.

Read on for questions to ask and how to translate that into a star schema model.

Comments closed

Interesting R Functions for Package Dependencies and File Analysis

Published 2023-08-31 by Kevin Feasel

Maelle Salmon shows off a few interesting functions:

How does this package depend on this other package? pak::pkg_deps_explain()

The pak package by Gábor Csárdi makes installing packages easier. If I need to start working on a package, I clone it, then run pak::pak() to install and update its dependencies. It’s a “convenience function” that is convenient for sure! Bye bye remotes::install_deps().

Read on for an example of this, as well as details on two other functions in different packages. H/T R-Bloggers.

Comments closed

Building Correlation Heatmaps in R

Published 2023-08-31 by Kevin Feasel

Steven Sanderson shows two packages for building heatmaps in R:

Data visualization is a powerful tool for understanding the relationships between variables in a dataset. One of the most common and insightful ways to visualize correlations is through heatmaps. In this blog post, we’ll dive into the world of correlation heatmaps using R, using the mtcars and iris datasets as examples. By the end of this post, you’ll be equipped to create informative correlation heatmaps on your own.

Read on to see how to build heatmaps with the corrplot and ggcorrplot packages.

Comments closed