Press "Enter" to skip to content

Author: Kevin Feasel

Getting Power BI Dataset Information without Takeover

Published 2021-10-19 by Kevin Feasel

Marc Lelijveld just wants to peek at some Power BI Dataset details:

We have all been in a situation where you want to get more insights in the specific settings for a published dataset. But when this dataset is owned and published by someone else, you have to take-over the dataset first to get all the details available. In this blog, I will elaborate on what is available to you in the service and share some code snippets that help you to explore other dataset settings like refresh schedules and configured parameter values.

Read on to see what kinds of details Marc means, as well as a way to do it without taking ownership of the Dataset.

Comments closed

Deferred Name Resolution and Its Discontents

Published 2021-10-19 by Kevin Feasel

Kendra Little gives us a detailed rundown of deferred name resolution in SQL Server:

Have you ever tried to create an object in SQL Server, but it failed due to a missing table, column, or other dependency? If so, you’ve hit a case where SQL Server doesn’t offer ‘deferred name resolution’.

While these errors can be helpful when you’ve made a typo or accidentally used the wrong database, this can sometimes be a big hassle when you are…

– Deploying database code to set up a partial environment

– Deploying database code from version control to an empty database to ensure the code is valid

In this post, I walk through common scenarios and test whether deferred name resolution works or not.

Check it out for what is probably the most detailed look at the topic I’ve ever seen.

Comments closed

Explaining SSISDB

Published 2021-10-19 by Kevin Feasel

Andy Mallon explains why SSISDB exists:

Back in my day, we stored SQL Server Integration Services (SSIS) packages in msdb, or the file system. Well, I’m neither dead nor retired, so that’s not an accurate statement–today is still “my time.” But when I was learning about SQL Server, those were the only two options. If you or your environment predate SQL Server 2012, then you might only be familiar with those two options.

In SQL Server 2012, Microsoft introduced the SSISDB. Since SQL Server Integration Services (SSIS) is its own service, it deserves its own database. In a service-oriented world, it is discouraged to put data for one service (the database engine) in the same database as another service (integration services).

Click through to see where it comes from and when you might need it.

Comments closed

Creating Delta Lake Tables in Azure Databricks

Published 2021-10-18 by Kevin Feasel

Gauri Mahajan takes us through creating new tables in a Delta Lake using Azure Databricks:

Delta lake is an open-source data format that provides ACID transactions, data reliability, query performance, data caching and indexing, and many other benefits. Delta lake can be thought of as an extension of existing data lakes and can be configured per the data requirements. Azure Databricks has a delta engine as one of the core components that facilitates delta lake format for data engineering and performance. Delta lake format is used to create modern data lake or lakehouse architectures. It is also used to build a combined streaming and batch architecture popularly known as lambda architecture.

Click through for the process.

Comments closed

Contrasting Kafka with Azure Service Bus

Published 2021-10-18 by Kevin Feasel

Ritam Das explains the differences between Apache Kafka and Azure Service Bus:

 It is important to note that Azure Service Bus is a traditional message broker and tailored to somewhat different use cases when compared to Kafka. Simply transferring between these two technologies is not an easy feat and would require overhauling your entire application. The comparison stops at both technologies being message brokers as under the hood they are fundamentally different. 

At a high level, ASB has high processing overhead per message, stronger guarantees around delivery and processing, and typically a “process once” model. Kafka has low overhead processing per message, fewer guarantees around delivery and processing, and typically a “publish once, process multiple times” model. To provide an explicit comparison, it would be best to understand the intended use case and proceed from there. 

Read on to understand the best uses for each technology, as well as sample calls using Python.

Comments closed

Moving Files Associated with Availability Groups

Published 2021-10-18 by Kevin Feasel

Eitan Blumin has a doozy of a short script:

Today, I’m sharing with you a cool Powershell script that basically implements the methodology necessary to move database files to a new location in AlwaysOn Availability Groups, without breaking HADR.

It’s based on a few very useful step-by-step guides on the topic such as this one and this one and this one. But it takes it a step further by being a single cohesive Powershell script that does everything end-to-end.

Well… Almost everything… The only thing it’s missing is somehow disabling any SQL Agent jobs that may be performing backups. I still haven’t figured out how to possibly automate such a thing, so you’d have to do that manually on your own.

Click through for instructions, notes, and warnings, as well as the script itself.

Comments closed

Architecting a Jenkins Replacement

Published 2021-10-18 by Kevin Feasel

Li Haoyi takes us through an internal Databricks tool for continuous integration:

Runbot is a bespoke continuous integration (CI) solution developed specifically for Databricks’ needs. Originally developed in 2019, Runbot incrementally replaces our aging Jenkins infrastructure with something more performant, scalable, and user friendly for both users and maintainers of the service. This blog post will explore the motivations behind developing Runbot, the core design decisions that went into it, and how we used it to greatly improve the experience of all the developers within the Databircks engineering organization.

It doesn’t look like the tool is available externally, but it’s an interesting read and helps understand some of the “why” behind the solution.

Comments closed