Press "Enter" to skip to content

Category: Notebooks

The State of SQL Notebooks

Deborah Melkin takes a look at the state of notebooks in the Microsoft ecosystem:

Azure Data Studio, as some of you know, has now been deprecated. That came out a while ago and the official deprecation was, I think, a couple months ago at this point. It’s all a blur, but needless to say, no more Azure Data Studio. I had gotten an email from someone who said they saw my presentation and they’d love to see more about it, especially with VS Code. Because there was an extension in VS Code for notebooks, and particularly something called .NET Interactive, which are polyglot notebooks, polyglot multi-language. All right, that was really cool and I had started addressing that, too, because it had just been introduced. It’s a really cool concept.

And then before I had a chance to put it together, new notice from Microsoft. Guess what’s being deprecated?

You guessed it. .NET Interactive notebooks. They went bye-bye. Great.

But wait, I hear SQL Notebook’s theme music?

Leave a Comment

Retrieving Materialized Lake View Lineage and Refresh Times

Meagan Longoria wants information:

Materialized lake views (MLVs) in Microsoft Fabric are an effective way to implement medallion architecture declaratively, but once you have a pipeline of MLVs in production, you need visibility into whether they’re current. Fabric’s MLV management area gives you a visual lineage and refresh history, but if you want to build automated alerting, logging, or custom tooling, you need to get that information programmatically. This post walks through one way to do that, using a small demo lakehouse built entirely in a Fabric notebook.

Click through for that demonstration.

Comments closed

Unit Testing DAX via Semantic Link

Jens Vestergaard writes a test:

Every BI developer has felt it. You change a measure, update a relationship, or rename a column in a semantic model, and then you spend the next hour clicking through report pages to check if something broke. Manual spot-checking is how most teams validate DAX today. It works until it does not.

I have been building and maintaining semantic models for years. The further I get into Fabric-based development, the more my models start to feel like production code. They power dashboards that drive decisions. They feed downstream pipelines. When something breaks, the blast radius is real. And yet, the testing story has always been: deploy, open the report, squint at the numbers.

That gap bothered me enough to do something about it.

Click through to see what Jens has done.

Comments closed

Combining Fabric Real-Time Intelligence, Notebooks, and Spark Structured Streaming

Arindam Chatterjee and QiXiao Wang show off some preview functionality:

Building event-driven, real-time applications using Fabric Eventstreams and Spark Notebooks just got a whole lot easier. With the Preview of Spark Notebooks and Real-Time Intelligence integration — a new capability that brings together the open-source community supported richness of Spark Structured Streaming with the real-time stream processing power of Fabric Eventstreams — developers can now build low-latency, end-to-end real-time analytics and AI pipelines all within Microsoft Fabric.

You can now seamlessly access streaming data from Eventstreams directly inside Spark notebooks, enabling real-time insights and decision-making without the complexity & tediousness of manual coding and configuration.

Click through to learn more.

Comments closed

More Spark Jobs, Fewer Notebooks

Miles Cole lays out an argument:

I’m guilty. I’ve peddled the #NotebookEverything tagline more than a few times.

To be fair, notebooks are an amazing entry point to coding, documentation, and exploration. But this post is dedicated to convincing you that notebooks are not, in fact, everything, and that many production Spark workloads would be better executed as a non-interactive Spark Job.

Miles has a “controversial claim” at the end that I don’t think is particularly controversial at all. I agree with pretty much the entire article, especially around the difficulties of testing notebooks properly.

Comments closed

Backing up a Microsoft Fabric Workspace

Gilbert Quevauvilliers finds a gap and fills it:

In the high-stakes world of data architecture, where downtime can cascade into real business disruptions, I’ve learned that even the most robust platforms have their blind spots. Just last month, while collaborating with a client’s Architecture team on their disaster recovery strategy, we uncovered a subtle but critical gap in Microsoft Fabric: while OneLake thoughtfully mirrors data across multiple regions by default, other workspace items—like notebooks, semantic models, and pipelines—aren’t directly accessible in a failover scenario without extra steps. For the nitty-gritty on Fabric’s built-in reliability features, check out this Microsoft Learn guide.

That’s the spark that led me down this rabbit hole, and in this post, I’ll walk you through a practical solution: a Python Notebook that automates backing up your entire Fabric workspace to OneLake and an Azure Storage Account for that extra layer of redundancy. Whether you’re prepping for the worst or just embracing the “better safe than sorry” mindset, this approach gives you portable, versioned copies you can restore quickly.

Click through for the notebook, as well as instructions on how to use it.

Comments closed

Data Visualization and Microsoft Fabric Notebooks

Meagan Longoria thinks about notebooks:

Lots of people have created Power BI reports, using interactive data visualizations to explore and communicate data. When Power BI was first created, it was used in situations that weren’t ideal because that was all we had as far as cloud-based tools in the Microsoft data stack. Now, in addition to interactive reports, we have paginated reports and notebooks. In this post, I’ll discuss when notebooks might be an appropriate visualization tool.

Click through for Meagan’s thoughts.

Comments closed

Migrating Azure Data Studio SQL Notebooks to VS Code Polyglot Notebooks

Haroon Ashraf gives us a somewhat unwieldy process:

As a SQL/BI developer, I want to run and store my SQL scripts and documentation efficiently in a Notebook as an alternative to using Azure Data Studio SQL Notebooks since Azure Data Studio is retiring soon. Read on to learn more about Visual Studio Code Polyglot Notebooks.

I liked the simplicity of having a SQL kernel in Azure Data Studio. Haroon shows how to work around it and get to roughly the same spot, but I do hope the SQL Server tools team is able to migrate that SQL kernel over to VS Code prior to Azure Data Studio’s ultimate demise.

Comments closed