A Fine Slice Of SQL Server
Gilbert Quevauvilliers does a bit of writing:
When I was exploring how to easily write to Delta Tables with a Python notebook, it took me a considerable amount of time to find out how to do this.
This is my learnings below, and from my point of view it makes it easy to write to a Lakehouse table, like what is done with a PySpark notebook.
Click through for one very important note, as well as the process.
Comments closedDan Liu makes an announcement:
Task flows feature is now generally available! Task flows streamline the design of your data solutions and ensure consistency between design and development efforts. It also allows you to navigate items and manage your workspace more easily, even as it becomes more complex over time.
Read on to see what’s available to us now.
Comments closedAditya Jain announces a preview:
Azure Key Vault support in Fabric Data connections is now in preview! With this capability, we are introducing a new concept called ‘Azure Key Vault references’ in Microsoft Fabric, using which, users can reuse their existing Azure key vault secrets for authentication to data source connections instead of copy-pasting passwords, slashing credential-management effort and audit risk.
Click through to see what works so far and the current limitations.
Comments closedAI is meant to help us automate boring tasks, and what could be more boring than creating documentation for your Power BI semantic models? It’s such a tedious task that most people don’t bother; there’s also an ecosystem of third party tools that do this job for you, and you can also build your own solution for this using DAX DMVs or the new-ish INFO functions (see here for a good example). That got me wondering: can you use Fabric Data Agents to generate documentation for you? And what’s more, why even generate documentation when you can just ask a Data Agent the questions that you’d need to generate documentation to answer?
For a simple scenario, Chris was able to get pretty solid results. As complexity grows, your mileage may vary.
Comments closedTyler Chessman is like an onion:
Building a multi-layer, medallion architecture using Fabric Real-Time Intelligence (RTI) requires a different approach compared to traditional data warehousing techniques. But even transactional source systems can be effectively processed in RTI. To demonstrate, we’ll look at how sales orders (created in a relational database) can be continuously ingested and transformed through a RTI bronze, silver, and gold layer.
Read on to see how.
Comments closedDennes Torres wants to know who is your daddy and what does he do:
If you check the text of the queries, at the end of the text you will find content like this:
OPTION (label = N”{“DatasetId”:”1269551b-bf26-47de-b0f0-974fa60f7b08″,”Sources”:[{“ReportId”:”01ab9208-399a-47ec-b444-d03633fc3e1d”,”VisualId”:”30ac676503a0bd357312″,”Operation”:”AutoPageRefresh”}]}”)
This has an interesting meaning:
- We can use this information to track the query lineage
- Applications can send lineage (or more) to SQL using OPTION (LABEL) statement
Click through to learn how you can use this information.
Comments closedKoen Verbeeck answers a question:
We’re a relatively small shop that has been using Power BI for our analytical needs for years now. We’re very pleased with the product, but the recent introduction of Microsoft Fabric has made us a bit anxious. When comparing Microsoft Fabric vs Power BI, it all seems very complex and we’re not even sure we need it. What will happen with Power BI? Will it be replaced with Fabric?
Click through for Koen’s advice and thoughts on the matter.
Comments closedPatrick LeBlanc is back with another compendium:
Welcome to the Fabric April 2025 Feature Summary! This update brings exciting advancements across various workloads, including Low-code AI tools to accelerate productivity in notebooks (Preview), session Scoped distributed #temp table in Fabric Data Warehouse (Generally Available) and the Migration assistant for Fabric Data Warehouse (Preview) to simplify your migration experience.
This one isn’t quite as long as last month’s release, but there are still a couple dozen entries.
Comments closedAndy Leonard doesn’t need to ask for permission:
The problem we are trying to solve is: Grant an Azure Data Factory permission to execute the ReST API method calls against artifacts in a Fabric workspace.
I begin visiting the Azure Portal. If you don’t have an Azure account, you can create one. You can do a lot of things in Azure for little money, but you need an account and that account needs to be secured by a working source of funds in case you leave a virtual machine running for days. Hypothetically. BE CAREFUL. DO NOT LEAVE A VM RUNNING FOR DAYS.
Good en passant advice. I once blew through a $250 Azure credit by laving an HDInsight cluster on for, uh, a few hours.
Comments closedGilbert Quevauvilliers writes a SQL statement:
I come from a TSQL background, so using SQL makes it easy for me to work with data.
There are multiple ways to use SQL in a PySpark notebook, and when I started using a Python notebook it was not so straightforward.
In this blog post I will show you how I use SQL Code.
As mentioned previously I am by no means an expert, I typically find a way that works, is fast and doesn’t consume a lot of capacity. If that works consistently for me then that is how I go about it.
Click through for the solution, which uses DuckDB. As such, the SQL syntax isn’t T-SQL—it’s more like psql. But it does do a great job of interacting with Parquet files and Delta tables.
Comments closed