Press "Enter" to skip to content

Month: February 2023

Migrating from Elasticsearch to Azure Data Explorer

Published 2023-02-03 by Kevin Feasel

Bhaskar Kakaraparthy does a logging switcharoo:

This article is an extension to an existing article to migrate data from Elastic Search to Azure Data Explorer (ADX) using Logstash pipeline as a step-step-step guide.  In this article, we will explore the process involved in migrating data from one source (ELK) to another (ADX) and discuss some of the best practices and tools available to make the process as smooth as possible.

Using Logstash for data migration from Elasticsearch to Azure Data Explorer (ADX) was a smooth and efficient process. With the help of ADX output plugin & Logstash, I was able to migrate approximately 30TBs of data in a timely manner. The configuration was straightforward, and the data transfer with ADX output plugin was quick and reliable. Overall, the experience of using ADX output plugin with Logstash for data migration was positive and I would definitely use it again for similar projects in the future.

Read on to see how.

Comments closed

Database Constraints in Postgres

Published 2023-02-03 by Kevin Feasel

Grant Fritchey does some data modeling:

PostgreSQL supports constraints much like any other database management system. When you need to ensure certain behaviors of the data, you can put these constraints to work. I’ve already used several of these in creating my sample database (available articles publicly on GitHub, in the CreateDatabase.sql file). I’ll explain those as we go. The constraints supported by PostgreSQL are:

Read on for the list, which includes one constraint which doesn’t have a direct analog in SQL Server.

Comments closed

Power BI Workspace Roles

Published 2023-02-03 by Kevin Feasel

Reza Rad shares some recommendations with us:

Power BI workspaces are not like the old days when we had Edit access and View access only. You have more options for roles in a workspace, and in my courses, I have found that many people have chosen the incorrect role without knowing what the role does. In this article, I’ll explain all the roles in the workspace, and what is the best way to set them up to have a secure workspace.

Click through for the article, as well as an accompanying video. Or a video and an accompanying article, if that’s how you roll.

Comments closed

The Power of Managed Identities in Azure

Published 2023-02-02 by Kevin Feasel

Koen Verbeeck waxes about managed identities:

This however presented me with the opportunity to review what users were actually necessary in the ETL. Turns out, not that many. In many cases, one Azure resource (for example, an Logic App) can use a managed identity to access another Azure resource (such as an Azure SQL DB). For those of you not familiar with the concept, a managed identity is basically a service principal in Azure AD with the same name as your resource. If your Azure Data Factory instance is called myADF, you’ll have an entry in AAD with the name myADF (it’s very much alike the SQL Server service account used on-premises). You can then assign role permissions to this managed identity.

Read on to see how you can use these managed identities to grant permissions without having to set (or reset or store) passwords.

Comments closed

Regionally Formatted Dates in Excel and Power BI

Published 2023-02-02 by Kevin Feasel

Allison Kennedy wrangles dates:

Dates are fundamental to pretty much every report. No matter what industry you work in, at some stage you’re going to work with dates in your reporting. This might be in the form of Semesters, Quarters, Seasons, Weeks, or just good old fashioned Dates. 

If you’re working with Power Query or Power BI, you should have a Date Table. In this post, I’m going to demonstrate how to work with Dates that can be tricky to format. 

Click through for examples of three common challenges when working with dates in Excel and Power BI.

Comments closed

Registering AKS Endpoints on Azure DNS

Published 2023-02-02 by Kevin Feasel

Denny Cherry notes that the DNS server is in another castle:


If you have an Azure environment when you have your DNS servers in a separate vNet from your new AKS environment you’ll notice that you get an error when deploying the AKS environment which looks something like this.

Agents are unable to resolve Kubernetes API server name. It’s likely custom DNS server is not correctly configured, please see https://aka.ms/aks/private-cluster#hub-and-spoke-with-custom-dns for more information.

The fix for this is actually pretty straightforward, but I’m going to give you a little background on why this happens.

Click through for the answer.

Comments closed

Power BI App Audiences and Workspaces

Published 2023-02-02 by Kevin Feasel

Marc Lelijveld looks at an update to Power BI:

Back in August 2022, Microsoft released a feature to have multiple audiences for Power BI Apps, or if you will deploying multiple versions of the same app where you can actively show and hide content for different audiences. A very welcome feature that has a big impact on the number of workspaces people require to have to reach the same goals. In this blog, I will elaborate on the impact of multiple audiences for Power BI Apps, as well as the impact on workspace governance.

Read on for Marc’s thoughts on the topic.

Comments closed

Automating Power BI Premium Dataset Backup

Published 2023-02-02 by Kevin Feasel

Gilbert Quevauvilliers shares the first part of a two-part series:

The first part in this 2-part series I am going to explain how configure the Azure Runbook so that you can then re-use it for multiple different Power BI datasets.

I am confident that most people have more than one dataset that needs to be backed up.

Before starting, please make sure that you have connected your Power Per User or Premium App Workspace to Azure Storage

Read on for a high-level overview of how to create a runbook in Powershell, as well as the runbook code.

Comments closed

Simplified Syntax for Scala 3

Published 2023-02-01 by Kevin Feasel

Anshika Agrawal shows a few examples of how Scala 3 is a bit easier to pick up than Scala 2:

Scala 3 is a remodel/refit for the scala language. It attracts developers due to its improved features such as simpler syntax, better type inference, improved error messages, and enhanced support for functional programming. In this article, we will compare the syntactical enhancement of Scala 2 & Scala 3. How indentation will help developers to write code efficiently and effortlessly.

Click through for some examples. On the whole, these are small but welcome changes in eliminating unnecessary code cruft.

Comments closed