A Fine Slice Of SQL Server
Gilbert Quevauvilliers performs a split:
When loading data, it is always important to load the data with performance and scalability in mind.
For lakehouse tables to return queries quickly and to scale it is essential to load your lakehouse tables with partitions.
What I am going to show you in my blog post today is how to load data into a Lakehouse table where the table will be automatically partitioned by Year/Month/Day.
Click through for the example.
Comments closedReitse Eskens performs a comparison:
This blog will be a quite short one compared to the other blogs as it’s more of an overview to show you the capacity of Fabric ingesting CSV files in their native format into a Lakehouse and ingesting SQL data into a table structure inside the Lakehouse. Simple, straightforward stuff without any form of modification. You could call it bronze, raw, ingestion, temp or whatever your preferred naming convention is.
Why is this important? Well, we still have source systems that can only output to files. Just as we still have customers running on SQL Server 2000, legacy or even antique systems are still running. And it’s important to know how much capacity you use when just ingesting data without any modification.
Read on for the two scenarios, giving you an idea of which one is faster. I’d be interested in a third option, which is reading from Parquet files. My initial expectation would be that it would be even faster and more efficient, depending on the structure of the data.
Comments closedKoen Verbeeck talks permissions:
We are starting a new analytics project in Microsoft Fabric, and our data will land in a warehouse. This is the first time we’re using Fabric, and we are wondering about the different options for sharing access to a warehouse we developed in a workspace.
Click through for more information on providing and limiting access to data in a Microsoft Fabric warehouse.
Comments closedWhen it was announced there was a collective cheer from Power BI source control advocates heard ’round the world. Since it’s preview release, Microsoft has also added GIT integration with Fabric workspaces. This makes it so easy to incorporate source control for all (or almost all) of your Fabric artifacts, including Power BI.
But what happens when your organization already has a mature CI/CD process in place using Azure DevOps? Do you really want to break from that pattern and have it controlled somewhere else? That’s what this post is about, using Azure DevOps CI/CD pipelines to deploy your Power BI Project files (.pbip).
I’m going to share my experience in hopes that it will save you some time if this is the route you need to take.
Read on for Angela’s experience. Note that this applies both to Microsoft Fabric as well as a Fabric-less Power BI.
Comments closedMarc Lelijveld covers a new bit of functionality in Microsoft Fabric:
Yesterday, Microsoft released a new option called “branch-out” that allows you to easily setup a new branch from an existing Fabric workspace. Obviously, this was already possible but involved a lot of manual work. With this new option, you can create your own feature branch to work in isolation before you commit your work to the central repository.
In this blog, I will deep dive more in this branch-out feature, how it works, including some things to keep in mind working with this feature.
Read on to learn more about the feature.
Comments closedKoen Verbeeck copies some data:
We have a couple of Snowflake databases and would like to have that data available in Microsoft Fabric as well. Is there an easy solution to get the data quickly in Fabric? We don’t have many technical people on staff, so writing complex ETL is not an option.
Read on for more information on how it works. Mind you, you’re probably still writing the T and some of the L after using mirroring.
Comments closedGilbert Quevauvilliers takes on a challenge:
I always enjoy a good challenge and I got it working! In this blog post I will use the same method where I am sending an email to scale up or scale down my Fabric Capacity.
The good news is that this works if the Capacity is paused or running (It might take a bit more time when running).
brb, sending Gilbert’s task an e-mail.
Actually, Gilbert does a good job in making sure that the sender has to be his e-mail address and not just some rando’s.
Comments closedMarc Lelijveld breaks out the green eyeshade:
P-SKUs, A-SKUs, EM-SKUs and now we also have F-SKUs… all these different capacities that are out there today each have their own specifics. Lately, I’ve been in a lot of conversations around Fabric capacities. There seems to be some unclarity around what you pay for in the end and how it compares to Power BI Premium capacities. Therefore, I thought, maybe this is the right time to write it down – besides the Microsoft documentation that is already out there.
In this blog I will elaborate on differences in purchasing, billing and buying the capacities. I will not deep dive in capacity metrics or how capacity units are consumed.
There’s a lot of good information in the article, especially if you’re looking to price out Microsoft Fabric in your organization.
Comments closedReza Rad gives people a primer on Apache Spark:
Microsoft Fabric runs some workloads under the Spark engine, but what is it really? In this article, I’ll take you through the question of what Spark is, What benefits it has, how it is associated with Fabric, what configurations you have, and other things you need to know about it.
Reza talks a bit about history, interaction with languages, etc. As a quick addition to the languages list, you can use .NET languages like F# and C# with Spark, though it does involve setting up dotnet/spark and there are some open questions about its future. And I’m not even sure you could get it to work with Microsoft Fabric.
Comments closedNikola Ilic shows us the current options:
Options, options, options…Having the possibility to perform a certain task in multiple different ways is usually a great “problem” to have, although very often not each option is equally effective. And, Microsoft Fabric is all about “options”…You want to ingest the data? No problem, you can use notebooks, pipelines, Dataflows, or T-SQL. Data transformation needed? No worries at all – again, you may leverage notebooks, T-SQL, Dataflows…Data processing, you asked? Lakehouse (Spark), Warehouse (SQL), Real-Time Intelligence (KQL), Power BI…The choice is yours again.
In a nutshell, almost every single task in Microsoft Fabric can be completed in multiple ways, and there is no “right” or “wrong” tool, as long as it gets the job done (of course, as efficiently as possible).
Nikola lays out two pre-requisites and then shows us two options we can currently use, and three potential options we currently cannot use.
Comments closed