Press "Enter" to skip to content

Author: Kevin Feasel

Cross-Cloud Data Replication with Confluent

Ahmed Saef Zamzam and Hannah Miao move some data:

Cross-cloud replication over private networks is powered by Cluster Linking, Confluent’s fully managed, offset-preserving replication service that mirrors topics across clusters. Cluster Linking already makes it simple to connect environments across regions, clouds, and hybrid deployments with near-zero data loss. Now, with private cross-cloud replication, the possibilities expand even further—enabling secure multicloud data sharingdisaster recovery, and compliance use cases that many organizations, particularly those in regulated industries, have struggled to solve for years.

Click through to see how it works and how it can beat mechanisms that existed prior to it.

Leave a Comment

Inferential Statistics in Excel using R

Adam Gladstone does a bit of inference testing:

In the previous posts in this series (Using R in Excel) I have demonstrated some basic use-cases where using R in Excel is useful. Specifically we have looked at descriptive statisticslinear regressionforecasting, and calling Python. In this post, I am going to look at inferential statistics and how R can be used (in Excel) to perform some typical statistical tests. Excel provides many excellent facilities for data wrangling and analysis. However, for certain types of statistical data analysis the limitations of the built-in functions and the Analysis ToolPak is not sufficient, and R provides superior facilities.

Read on for a few examples of tests, though there are a huge number available in R itself as well as its ecosystem of packages.

Leave a Comment

Dealing with Many-to-Many Relationships in Power BI

Boniface Muchendu handles a many-to-many relationship:

Many-to-many relationships in Power BI are one of the most frequent challenges faced by new and intermediate users. These relationships can cause incorrect totals in visuals, confusing results, and slower report performance. In this guide, we’ll explore what many-to-many relationships in Power BI are, why they’re problematic, and how to fix them using the most effective methods available.

Click through for an enumeration of the problem as well as a couple of ways to resolve it.

Leave a Comment

Table Switching in SQL Server

Rebecca Lewis swaps in a bag of sand while snatching the golden idol:

Use ALTER TABLE … SWITCH to move very large tables instantly.  Yes.  I said instantly.  ALTER TABLE .. SWITCH doesn’t copy the data or physically move it.  It just reassigns the page ownership.  This means that only the metadata with the data pointer changes, and that’s why it completes in milliseconds and barely touches the transaction log.

There are some strict conditions around when you can use this technique, but it’s as powerful as Rebecca mentions.

Leave a Comment

User-Defined Input Screens for Power Apps

Jon Vöge lets users choose their own adventures:

To give a more specific example, I faced this requirement with a customer who used a Power App for Key Account Management data.

Key Accounts would be sorted into different categories, and each category of Key Accounts had slight variations in the input fields required. And the real kicker was that new Categories of accounts would frequently be added, meaning a new set of questions for this new category of accounts had to be defined.

In the ideal world, I wanted to remove the Power Apps developer from the equation, and let the users themselves define new questions as the need arises.

How would you solve this?

Click through for a solution using Power Apps.

Leave a Comment

API Interaction with OneLake Tables

Matthew Hicks makes an announcement:

Microsoft OneLake is the unified data lake for your entire organization, built into Microsoft Fabric. It provides a single, open, and secure foundation for all your analytics workloads – eliminating data silos and simplifying data management across domains.

The preview of Microsoft OneLake Table APIs, a new way to programmatically manage and interact with your data tables in OneLake! These APIs open the door for developers and data engineers to integrate OneLake seamlessly into their workflows, enabling powerful automation and interoperability with open table formats.

Read on to see what’s available in the initial preview. It’s interesting that they started with Iceberg rather than Delta Lake.

Leave a Comment

Diagnosing a Partition Job Failure after Migration to an AG

Mike Lynn describes a customer issue:

Quick Summary

A client noticed one of their reporting tables wasn’t logging any new information after the first of the new month.

Context

This environment ran on SQL Server 2019 in an Always On Availability Group configuration hosted on AWS EC2 servers. This is roughly 30-45 days after the servers were migrated from a SQL Server Failover Cluster Instance in AWS on EC2 to the new AG setup.

Read on for the problem, the discovery process, and the solution. I like reading this sort of report specifically to focus on the process. One of the best skills you can develop in any technical field is the practice of methodical behavior: review and understand the error message (perhaps with the assistance of a search engine or tool of choice), then work logically through possible issues until you discover the cause. It sounds obvious when I describe it that way, but far too often, people flail about and try a variety of arbitrary things because they don’t really understand the issue and hope that doing this one thing will fix whatever problem is happening.

Leave a Comment

Trying the Graph Database in Microsoft Fabric

Chris Webb gives the graph database in Microsoft Fabric a try:

The new Fabric Graph database is now rolling out now and should be available to everyone within the next few weeks if you can’t see it already. The key to learning a new data-related technology is, I think, to have some sample data that you’re interested in analysing. But if you’re a Power BI person why would a graph database be useful or interesting? Actually I can think of two scenarios: analysing dependencies between DAX calculations and the tables and columns they reference using the data returned by INFO.CALCDEPENDENCY function (see here for more details on what this function does); and the subject of this blog post, namely analysing Import mode refresh job graphs.

Read on for Chris’s example.

Leave a Comment

Reviewing Security Drift in SQL Server Reporting Services

Andy Brownsword checks out who has access to what permissions:

Item level roles are what we’re digging into here. Before we start, it’s worth defining a simple security model so it’s applied consistently. Let’s be real, the instance might not have a long term future but let’s do it right at least, eh?

Read on for a few high-level suggestions, details on what permissions do not carry over from parent objects, and more.

Leave a Comment