Press "Enter" to skip to content

Category: Notebooks

Combining Fabric Real-Time Intelligence, Notebooks, and Spark Structured Streaming

Published 2026-02-06 by Kevin Feasel

Arindam Chatterjee and QiXiao Wang show off some preview functionality:

Building event-driven, real-time applications using Fabric Eventstreams and Spark Notebooks just got a whole lot easier. With the Preview of Spark Notebooks and Real-Time Intelligence integration — a new capability that brings together the open-source community supported richness of Spark Structured Streaming with the real-time stream processing power of Fabric Eventstreams — developers can now build low-latency, end-to-end real-time analytics and AI pipelines all within Microsoft Fabric.

You can now seamlessly access streaming data from Eventstreams directly inside Spark notebooks, enabling real-time insights and decision-making without the complexity & tediousness of manual coding and configuration.

Click through to learn more.

Leave a Comment

More Spark Jobs, Fewer Notebooks

Published 2026-02-05 by Kevin Feasel

Miles Cole lays out an argument:

I’m guilty. I’ve peddled the #NotebookEverything tagline more than a few times.

To be fair, notebooks are an amazing entry point to coding, documentation, and exploration. But this post is dedicated to convincing you that notebooks are not, in fact, everything, and that many production Spark workloads would be better executed as a non-interactive Spark Job.

Miles has a “controversial claim” at the end that I don’t think is particularly controversial at all. I agree with pretty much the entire article, especially around the difficulties of testing notebooks properly.

Leave a Comment

Backing up a Microsoft Fabric Workspace

Published 2025-12-04 by Kevin Feasel

Gilbert Quevauvilliers finds a gap and fills it:

In the high-stakes world of data architecture, where downtime can cascade into real business disruptions, I’ve learned that even the most robust platforms have their blind spots. Just last month, while collaborating with a client’s Architecture team on their disaster recovery strategy, we uncovered a subtle but critical gap in Microsoft Fabric: while OneLake thoughtfully mirrors data across multiple regions by default, other workspace items—like notebooks, semantic models, and pipelines—aren’t directly accessible in a failover scenario without extra steps. For the nitty-gritty on Fabric’s built-in reliability features, check out this Microsoft Learn guide.

That’s the spark that led me down this rabbit hole, and in this post, I’ll walk you through a practical solution: a Python Notebook that automates backing up your entire Fabric workspace to OneLake and an Azure Storage Account for that extra layer of redundancy. Whether you’re prepping for the worst or just embracing the “better safe than sorry” mindset, this approach gives you portable, versioned copies you can restore quickly.

Click through for the notebook, as well as instructions on how to use it.

Comments closed

Data Visualization and Microsoft Fabric Notebooks

Published 2025-11-24 by Kevin Feasel

Meagan Longoria thinks about notebooks:

Lots of people have created Power BI reports, using interactive data visualizations to explore and communicate data. When Power BI was first created, it was used in situations that weren’t ideal because that was all we had as far as cloud-based tools in the Microsoft data stack. Now, in addition to interactive reports, we have paginated reports and notebooks. In this post, I’ll discuss when notebooks might be an appropriate visualization tool.

Click through for Meagan’s thoughts.

Comments closed

Migrating Azure Data Studio SQL Notebooks to VS Code Polyglot Notebooks

Published 2025-08-27 by Kevin Feasel

Haroon Ashraf gives us a somewhat unwieldy process:

As a SQL/BI developer, I want to run and store my SQL scripts and documentation efficiently in a Notebook as an alternative to using Azure Data Studio SQL Notebooks since Azure Data Studio is retiring soon. Read on to learn more about Visual Studio Code Polyglot Notebooks.

I liked the simplicity of having a SQL kernel in Azure Data Studio. Haroon shows how to work around it and get to roughly the same spot, but I do hope the SQL Server tools team is able to migrate that SQL kernel over to VS Code prior to Azure Data Studio’s ultimate demise.

Comments closed

Populating Microsoft Fabric Data Agents with Semantic Model Synonyms

Published 2025-06-04 by Kevin Feasel

Marc Lelijveld explains some terms:

It was only yesterday, that I wrote a blog post on Semantic Models as a source for Fabric Data Agents. Not much time has passed, since I learned that Fabric Data Agents does not (always) respect the Synonyms that have been added to a Semantic Model. As a result, the Data Agent may start creating implicit measures, not respecting the definitions and logic in the explicit measures that are part of the Semantic Model.

Long story short, I think we should be able to do better! Therefore, I created a Notebook that helps you to setup Data Agents, collect additional information from your Semantic Model and populate that information automatically as AI notes to the Data Agent.

Read on for the notebook and some additional explanation.

Comments closed

Writing a Python Data Frame to a Lakehouse Table

Published 2025-06-04 by Kevin Feasel

Gilbert Quevauvilliers continues a series on Python notebooks and DuckDB:

In this blog post I am going to explain how to loop through a data frame to query data and write once to a Lakehouse table.

The example I will use is to loop through a list of dates which I get from my date table, then query an API, append to an existing data frame and finally write once to a Lakehouse table.

Click through for the code, as well as a sample notebook you can use.

Comments closed