Press "Enter" to skip to content

Category: Microsoft Fabric

Ingesting Logs into Microsoft Fabric Real-Time Intelligence via Logstash

Published 2025-07-15 by Kevin Feasel

Surya Teja Josyula and Ramachandran G. use one part of the ELK stack:

Logstash is an open-source data processing tool that enables the collection, transformation, and forwarding of data from a wide variety of sources. It acts as a data pipeline engine, helping organizations manage and streamline the flow of structured and unstructured data across systems.

Whether you’re managing infrastructure logs, application events, or telemetry data, this guide will walk you through setting up a seamless pipeline that bridges raw log data with real-time analytics in Fabric.

Click through for the process.

Comments closed

Accessing Delta Lake Tables as Iceberg Data

Published 2025-07-11 by Kevin Feasel

Matthew Hicks makes an announcement:

We’re thrilled to announce an exciting new Preview capability in OneLake: you can now automatically read Delta Lake tables using Apache Iceberg compatible readers, with no need for migration, copying, or manual conversion. This enhancement gives data engineers and analytics teams unprecedented flexibility in how they access and interact with their data.

This is pretty neat, given that Iceberg is the other popular format for data lakes.

Comments closed

Goodbye Default Contributor Role in Fabric Workspace Identities

Published 2025-07-10 by Kevin Feasel

Varun Jain makes a security announcement:

A Fabric workspace identity is an automatically managed service principal that can be associated with a Fabric workspace. Fabric workspaces with a workspace identity can securely read or write to firewall-enabled Azure Data Lake Storage Gen2 accounts through trusted workspace access for OneLake shortcuts. Fabric items can use the identity when connecting to resources that support Microsoft Entra authentication. Fabric uses workspace identities to obtain Microsoft Entra tokens without the customer having to manage any credentials. 

Previously, a workspace identity was automatically assigned the workspace contributor role and had access to workspace items.  

Read on to see what’s changing, why, and what you can do instead.

Comments closed

Writing Back to a Fabric Data Warehouse via UDF

Published 2025-07-08 by Kevin Feasel

Jon Vöge continues a series on write-back options into Microsoft Fabric:

In that article, we took advantage of some of the built-in sample code from the User Data Function editor, as well as some great code examples from Sujata: Example User data functions for Translytical task flows · GitHub

The problem? All of these samples use SQL Databases in Fabric as the backend item.

Jon switches this from a SQL database into a Fabric Data Warehouse, and notes some of the challenges along the way.

Comments closed

Event Streaming in Microsoft Fabric

Published 2025-07-08 by Kevin Feasel

Rayis Imayev streams some data:

In my post last week (https://datanrg.blogspot.com/2025/06/salesforce-cdc-data-integration.html), I talked about Salesforce Change Data Capture (CDC) event data streaming, where the initial event destination was file storage in Azure. But what if we anticipate a higher volume of incoming Salesforce source data or the addition of a new data feed? This could create the need for an alternative method of managing incoming events.

Read on to learn more.

Comments closed

Trying out Fabric Unified Admin Monitoring

Published 2025-07-07 by Kevin Feasel

Reitse Eskens tries out a tool for monitoring Microsoft Fabric installations:

Let me set the scene quickly for you. You’re working for an organisation where Fabric is in the process of being adopted or it’s already fully in use. Regardless of the number of capacities, workspaces, etc, you’re interested in what’s going on in your Fabric environment. You have questions like “Who is using the reports?”, “Who is changing settings in the Admin panel?” and “How is my capacity being used?”.

Read on for a single tool that can solve these sorts of questions.

Comments closed

The Small Data Showdown in Microsoft Fabric

Published 2025-07-01 by Kevin Feasel

Miles Cole does a bit of testing:

First, let’s revisit the purpose of the benchmark: The objective is to explore data engineering engines available in Fabric to understand whether Spark with vectorized execution (the Native Execution Engine) should be considered in small data architectures.

Beyond refreshing the benchmark to see if any core findings have changed, I do want to expand in a few areas where I got great feedback from the community:

I really appreciate the approach behind this, both in terms of sticking to more realistic data sizes for many operations as well as performing this test given all of the recent improvements in each engine.

Comments closed

Incremental Copy Job in Microsoft Fabric now GA

Published 2025-07-01 by Kevin Feasel

Ye Xu has an announcement:

Copy job has been a go-to tool for simplified data ingestion in Microsoft Fabric, offering a seamless data movement experience from any source to any destination. Whether you need batch or incremental copying, it provides the flexibility to meet diverse data needs while maintaining a simple and intuitive workflow.

We continuously refine Copy job based on customer feedback, enhancing both functionality and user experience. In this update, we’re introducing several key improvements designed to streamline your workflow and boost efficiency.

Click through to see what’s new.

Comments closed

Customer Managed Keys in OneLake

Published 2025-06-27 by Kevin Feasel

Harmeet Gill shows us how we can bring our own keys to data in OneLake:

One of the highly requested features in Microsoft Fabric is now available: the ability to encrypt data in OneLake using your own keys. As organizations face growing data volumes and tighter regulatory expectations, Customer-Managed Keys (CMK) offer a powerful way to enforce enterprise-grade security and ensure strict ownership of encryption keys and access.

With Microsoft’s OneLake, we’ve built a unified data lake that’s open, secure, and ready for enterprise scale. Now, with support for CMK, we’re giving customers the power to take encryption into their own hands.

Read on to learn more about Microsoft’s default for data encryption, and how you can use your own keys to encrypt the data.

Comments closed