A Fine Slice Of SQL Server
Sam Debruyn looks at what lurks inside a Spark pool:
Microsoft Fabric has been out for a couple of years now. Have you ever wondered what is under the hood? There are lots of interesting things hiding in plain sight. Today, let’s take a look at Project Vegas, the intelligent caching layer for Spark in Synapse and Fabric.
Click through for the results of Sam’s dive.
Leave a CommentChris Webb notes that parallelism does not mean “free performance improvements”:
To finish off my series of posts on concurrent evaluation in Fabric Dataflows Gen2 (see part 1 and part 2) I decided to do some more realistic tests to see how much parallelism I could get. To do this I uploaded 244 identical Excel files containing almost 542000 rows of data each to a SharePoint document library. Excel files are probably the worst-performing file format for dataflows (see here for some tests that show this), while SharePoint is probably the worst-performing place to store data for a dataflow and also has a reputation for throttling applications that make too many requests.
Click through for Chris’s test results.
Leave a CommentGilbert Quevauvilliers uses the private endpoint:
In this blog post I show how it is possible to access data between a Private Link Workspace, where I want to read the data from my Public Workspace.
An example of this is where I wanted to use a DirectLake Semantic Model sitting in my Public Workspace where the data is from data in my Private Link Workspace.
Click through to see how it works.
Leave a CommentKurt Buhler explains a new capability:
For years, the Power BI community has been clamoring to have native support for visuals-as-code; the ability to create visuals, pages, or even entire dashboards with libraries like Vega, and D3.js. This is now possible in Microsoft Fabric with Fabric Apps, specifically, using a data app.
A Fabric App is a new item type that lets you create and distribute any interactive experiences in Fabric, rather than pre-defined reports, dashboards, and data agents. These are web applications (or webapps). A webapp is any program that runs in a browser (like YouTube, Facebook, or Microsoft Word’s online editor) instead of one that you install on your computer (like Power BI Desktop or Microsoft Word).
Read on to see how you can build Fabric Apps, as well as what they are and aren’t.
Leave a CommentKoen Verbeeck troubleshoots an issue:
A while ago we suddenly had an error while trying to deploy one Fabric workspace to another using fabric-cicd. The issue was with a GraphQL object and the following error was returned:
Failed to publish GraphQLApi ‘my_graphql’: Operation failed. Error Code: DatasourceInvalidStoredProcedure. Error Message: Only those stored procedures whose metadata for the first result set described by sys.dm_exec_describe_first_result_set are supported.
Read on for Koen’s diagnosis and resolution for this issue.
Leave a CommentMarc Lelijveld digs into Microsoft Fabric Data Agents:
What makes Fabric Data Agents particularly interesting is the wide range of supported data sources. Today, Data Agents can connect to nearly everything that lives inside Microsoft Fabric, or data that is linked into Fabric through shortcuts. Whether your data sits in a Lakehouse, Warehouse, KQL Database, Power BI semantic model, or even external storage connected through OneLake shortcuts.
However, the way Data Agents handle sources can differ significantly from one source type to another. Semantic Models in particular behave quite differently compared to other Fabric data sources. In this blog, I’ll dive deeper into how prompt handling works for Semantic Models, what happens behind the scenes, and the common gotchas you’re likely to encounter along the way.
Read on to see how semantic model behavior differs in particular from SQL or DAX queries.
Leave a CommentReitse Eskens hits the recycle bin:
Let’s be honest: how many times have you accidentally deleted something? Either on your laptop, in a database or in a SaaS product.
It happens. We’re all humans (unless you allow agents to do all your work for you), and mistakes happen.Until recently, when you deleted an item in Fabric, it was gone. Poof. Done. No grace period.
And that was a bit scary, to be honest, but now we have a new option to help us recover from oopsies!
The answer to Reitse’s question is “far too often for me to want to admit out loud.”
Leave a CommentChris Webb continues to test out Fabric Dataflow Gen2 performance:
In the first part of this series I showed how the Concurrency setting in a Fabric Dataflows Gen2 can affect refresh performance when there are multiple queries inside the dataflow. In this post I will show how, with Partitioned Compute, this setting can also affect the performance of a single query within a dataflow.
To test this I created a dataflow with one query, a modified version of the query that I used in this post from earlier this year which returns a table with ten rows and calls a function with a built-in delay of 60 seconds on each row.
This is a preview feature but Chris shows a simple but effective test to demonstrate how this capability works.
Leave a CommentGilbert Quevauvilliers locks down a Fabric environment:
In this blog post, I am going to walk through how I created a Private Link in Microsoft Fabric so that access routes over Microsoft’s private backbone network instead of the public internet.
NOTE: It is a bit of a longer blog post as there are quite a few things that need to be configured.
NOTE II: In the next blog post I will show you how to connect from a Public Workspace to a Private Link Workspace using a Managed Private Endpoint
I wanted to document the end-to-end process because there are a few moving parts across Microsoft Fabric and the Azure Portal. The key items I needed to configure were the Fabric tenant setting, the Fabric private link service, the virtual network, a test virtual machine, the private endpoint, and finally DNS testing from inside the virtual network.
There’s a lot more here than ticking a couple of checkboxes or selecting a few radio button options.
Leave a CommentChris Webb runs multiple jobs at once:
Did you know that if your Fabric Dataflows Gen2 contains several queries then you can control how many of them are evaluated in parallel when your dataflow refreshes? In this series I’ll look at how how you can do this and how it may result in better performance – at least in some cases.
Let’s start with the basics. I created a Dataflow Gen2 with ten queries which each returned a table of one row and one column after one minute. I used the #table function to generate the table without connecting to a data source, code from this post to add the delay and the trick in this post to make sure the delay was only applied when the dataflow refreshed. The output of each query was loaded to a Fabric Warehouse.
Click through for a demonstration.
Leave a Comment