A Fine Slice Of SQL Server
Gilbert Quevauvilliers builds a data frame:
In this blog post I am going to explain how to query multiple Lakehouse tables into a data frame.
The example I am going to use is when you want to load new data into your staging tables, but you need to know the max date from your previous data load.
Read on to see how. The answer, as you might suspect, involves DuckDB.
Leave a CommentGilbert Quevauvilliers does a bit of writing:
In this blog post I am going to explain how to use a Python Notebook using the Semantic Link module, to run a DAX query and write the output to a Lakehouse table.
I will show you how to install a Python library and then use it within my python notebook.
Read on for a quick primer on Semantic Link Labs, followed by the meat of the article.
Leave a CommentOlivier Van Steenlandt runs into an error:
As mentioned earlier, I have been playing around with Microsoft Fabric intensively in the past few months. During this period, I ran into a specific issue with one of my notebooks. What happened? Well, I was starting on a new notebook in the evening and life happened… So I stopped playing around to do something else.
A few days later, I wanted to continue my work and remembered that I was required to change something in my data load from a csv file.
Read on for the cause of this error. It’s something that can affect anyone at any time. Even you. Well, probably not you, but the person next to you? Yeah, even that person.
Leave a CommentWorking with sensitive data in Microsoft Fabric requires careful handling of secrets, especially when collaborating externally. In a recent customer engagement, I needed to validate access to Azure Key Vault from within a Fabric Notebook, without ever exposing the actual secret values. With only read access granted and no need to manage or update secrets, I focused on confirming that the connection was working as expected.
In this blog, I’ll walk you through the approach, including the setup, code snippets, and logic behind this quick but crucial verification step.
Click through for the full story.
Comments closedGilbert Quevauvilliers does a bit of writing:
When I was exploring how to easily write to Delta Tables with a Python notebook, it took me a considerable amount of time to find out how to do this.
This is my learnings below, and from my point of view it makes it easy to write to a Lakehouse table, like what is done with a PySpark notebook.
Click through for one very important note, as well as the process.
Comments closedMarc Lelijveld provides a public service announcement:
When working with Notebooks in Microsoft Fabric, exporting and reusing them across environments or tenants might seem like a harmless, even convenient, task. Whether you’re sharing a template with a colleague, moving assets between workspaces, or contributing to the community — the last thing you’d expect is to accidentally include data along with your code.
But that’s exactly what can happen.
For people who have worked with Jupyter notebooks in the past, this is a fairly obvious result. But if you aren’t familiar with the platform, that idea may seem weird. Marc does provide some options for exporting notebook contents, and you can also clear the cell contents before exporting.
Comments closedMiles Cole picks up from part one:
This is part 2 of my prior post that continues where I left off. I previously showed how you can use Resource folders in either the Notebook or Environment in Microsoft Fabric to do some pretty agile development of Python modules/libraries.
Now, how exactly can you package up your code to distribute and leverage it across multiple Workspaces or Environment items? How could we acomplish something like the below?
Read on for the answer.
Comments closedDennes Torres wants to hit a SQL endpoint:
Let’s analyse why we would like to query an SQL Endpoint. Once we understand why, we can dig into how to make a query to a SQL Endpoint.
We use notebooks to connect directly to lakehouse. Except by the T-SQL notebook, the notebooks have a default lakehouse and work directly with it from spark. However, accessing other data object may be more complex
Specifically, this is a Spark notebook in Microsoft Fabric running Scala rather than a pure Python notebook, and is hitting the data warehouse SQL endpoint.
Comments closedSince Lakehouse is one of the key items within Microsoft Fabric, it is important to know how to write data into it in various formats and using different tools. One of the most common tools is notebooks, as they provide great flexibility and speed for development and testing with graphical outputs. In this article, I want to focus primarily on the following types of notebooks:
- PySpark
- Python
Click through to see how it works in both notebook types.
Comments closedGilbert Quevauvilliers doesn’t need Spark for this:
This blog below explains how to use a Python only notebook to get all the Fabric items using the Fabric REST API.
NOTE: At the time of this blog post Feb 2025, Dataflow Gen2 is not included in the Fabric items, I am sure it will be there in the future.
NOTE II: This only gets the Fabric Items, which does not include the Power BI Items.
Despite the notes, Gilbert leads off with the main reason why you might want to use this: it takes up approximately 5% of the capacity units that a Spark-based notebook does to perform the same operation.
Comments closed