A Fine Slice Of SQL Server
Sandeep Pawar digs into the REST API:
Accessing Fabric REST endpoints in Fabric notebooks was already easy but it became easier and straightforward with semantic-link version 0.4.0. You can use the FabricRestClient class from sempy to set up a REST client and call the APIs. Authentication is automatically managed for you.
Click through to see how it works, as well as some warnings or things to keep in mind along the way.
Comments closedI have published videos and articles before about Lakehouse maintenance. In this article I want to address a missing point for a lot of Fabric administrators: How to do maintenance on multiple lakehouses that are located in different workspaces.
One of the videos I have published explains the maintenance of multiple lakehouses, but only addresses maintenance in a single workspace. Is it a good idea to keep multiple lakehouses in the same workspace? Probably not.
Click through for the process.
Comments closedA few days ago, while preparing materials for the customer training on Microsoft Fabric, I stumbled upon a very interesting article at Microsoft Learn. The article describes how to integrate Power BI semantic models (aka datasets) into OneLake.
At first glance, this doesn’t sound like something epic, but when I started thinking more and more about it, I realized that this really might be a huge thing in many different scenarios. First of all, at the moment of writing, this feature is still in preview – this means, it can change to some extent in the coming months, before eventually becoming GA. Nevertheless, I decided to take a shot and explore what can be done with OneLake integration for semantic models.
Read on to learn more about what this is doing and what you can do with it.
Comments closedEugene Meidinger walks onto a field of rakes:
Last week, I struggled to load the data into Fabric, but finally got it into a Lakehouse. I was starting to run into a lot of frustration, and so it seemed like a good time to back up and get more oriented about the different pieces of Fabric and how they fit together. In my experience, it’s often most effective to try to do something, review some learning, and alternate. Without a particular pain point, it’s hard for the information to stick.
Read on for some thoughts on andragogy, learning paths, and travails loading data.
Comments closedTomaz Kastrun continues a series on Microsoft Fabric. Day 15 covers building a warehouse:
I have named my as “Advent2023_DWH”.
You can create a warehouse using T-SQL scripts, from data flow gen2, from data pipelines and from the sample data. Let’s select the sample data and grab a coffee.
Day 16 looks at data pipelines:
With the Fabric warehouse created and explored, let’s see, how we can use pipelines to get the data into Fabric warehouse.
In the existing data warehouse, we will introduce new data. By clicking “new data”, two options will be available; pipelines and dataflows. Select the pipelines and give it a name.
And Day 17 provides a primer on how Power BI can read Fabric assets:
Comments closedWithin the Power BI in Fabric, you will find many of the components, that can be used to create a final report. And here are the components:
Tomaz Kastrun continues a series on Microsoft Fabric. First up is creating ML models:
Protip: Both experiments and the ML model version look similar, and you can intuitively switch between both of them. But do not get confused, as the ML Model version applies the best-selected model from the experiment and can be used for inference.
Then we switch context to data warehousing:
Comments closedToday we will start exploring the Fabric Data Warehouse.
With the data lake-centric logic, the data warehouse in Fabric is built on a distributed processing engine, that enables automated scaling. The SaaS experience creates a segway to easier analysis and reporting, and at the same time gives the ability to run heavy workloads against open data format, simply by using transact SQL (T-SQL). Microsoft OneLake gives all the services to hold a single copy of data and can be consumed in a data warehouse, datalake or SQL Analytics.
Eugene Meidinger continues learning about Azure Fabric:
I want to preface that a lot of the issues I run into below are because of my own ignorance around the tooling, and a lot of the detail I include is to show what that ignorance looks like, since many people reading this might be used to Fabric or at least data engineering.
Read on to learn about Eugene’s mistakes so you don’t make them yourself.
Comments closedPhil Seamark tries out Vega in a Microsoft Fabric notebook:
I recently needed to generate a quick visual inside a Microsoft Fabric notebook. After a little internet searching, I found there are many good quality charting libraries in Python, however it was going to take too long to figure out how to create a very specific type of chart.
This is where Vega came to the rescue. The purpose of this short article is to share a very simple implementation of generating a Vega chart using a Microsoft Fabric notebook.
Click through for the example code.
Comments closedTomaz Kastrun has been busy. On day 9, we build a custom environment:
Microsoft Fabric provides you with the capability to create a new environment, where you can select different Spark runtimes, configure your compute resources, and create a list of Python libraries (public or custom; from Conda or PyPI) to be installed. Custom environments behave the same way as any other environment and can be used and attached to your notebook or used on a workspace. Custom environments can also be attached to Spark job definitions.
On day 10, we have Spark job definitions:
An Apache Spark job definition is a single computational action, that is normally scheduled and triggered. In Microsoft Fabric (same as in Synapse), you could submit batch/streaming jobs to Spark clusters.
By uploading a binary file, or libraries in any of the languages (Java / Scala, R, Python), you can run any kind of logic (transformation, cleaning, ingest, ingress, …) to the data that is hosted and server to your lakehouse.
Day 11 introduces us to data science in Fabric:
We have looked into creating the lakehouse, checked the delta lake and delta tables, got some data into the lakehouse, and created a custom environment and Spark job definition. And now we need to see, how to start working with the data.
We have started working with the data and now, we would like to create and submit the experiment. In this case, MLFlow will be used here.
Create a new experiment and give it a name. I have named my “Advent2023_Experiment_v3”.
Click through to catch up with Tomaz.
Comments closedTeo Lachev sees some potential:
I might have identified at last a good case for Microsoft Fabric, but I’ll be in a better position to confirm after a POC with larger datasets. Dynamics Online, aka Dynamics 365, epitomizes the customer’s struggle to export their data hosted in SaaS cloud offerings for analytics or other purposes. Since unfortunately Microsoft doesn’t provide direct access to the Dynamics native storage (Azure SQL Database), which often could be the simplest and fastest solution, Dynamics has “compensated” throughout the years by introducing and sunsetting various options:
Click through for that table, as well as Teo’s thoughts on the possibility of Dataverse Link to Fabric as a viable solution.
Comments closed