Press "Enter" to skip to content

Category: Microsoft Fabric

Reading a Lakehouse Table from another Microsoft Fabric Workspace

Published 2024-08-07 by Kevin Feasel

Gilbert Quevauvilliers spans the gap:

I was doing some work recently for a customer and they had data stored in different Lakehouse’s which was in a different App Workspace.

I was pleasantly surprised that this can be quite easy to do.

In my example below I am going to show you how in my notebook I can read a table in a Lakehouse table when it is not attached to any Lakehouse.

It’s good that this is so easy to do, considering that current advice leans toward having multiple workspaces and not cramming everything into one.

Comments closed

Defining a OneLake Filesystem using fsspec

Published 2024-08-05 by Kevin Feasel

Sandeep Pawar looks at fsspec:

I mentioned on X the other day that, like other filesystem backends such as S3 and GCS, you can use fsspec to define the OneLake filesystem too. In this blog, I will explain how to define it and why it’s important to know about it.

Click through for the details on what fsspec is, why it’s important, and what benefits you can get in Microsoft Fabric as a result of its support of fsspec.

Comments closed

Security Options in Microsoft Fabric Warehouses

Published 2024-07-30 by Kevin Feasel

Koen Verbeeck locks things down:

We are implementing a data analytics solution in Microsoft Fabric. A warehouse is used for the gold layer, and we want to give users access to the data. However, by sharing the warehouse, they can read all the data in all the tables. Some data is sensitive, and only users with the correct permissions should be able to view it. Is it possible to implement more granular access control to the data?

Read on for the answer, as well as an important note on how users might be able to circumvent your permissions settings.

Comments closed

Microsoft Fabric GitHub Integration Security Considerations

Published 2024-07-30 by Kevin Feasel

Kevin Chant covers a bit of security:

I know the option to work with GitHub has got a lot of people excited. Which I why wanted to share my initial thoughts about security with you all. Because a lot of things have come to mind whilst testing this.

I want to highlight immediate implications and options before you all get too involved with testing. To make sure you test working with GitHub safely.

Plus, this post is really useful for those of you looking to test this in a regulated GitHub Enterprise environment. Because it will allow you to explain things to your GitHub administrators better, and/or forward them this post. To explain what you want to achieve.

Read on for Kevin’s thoughts on the matter.

Comments closed

FabricRestClient and Long-Running Operations

Published 2024-07-29 by Kevin Feasel

Sandeep Pawar has a public service announcement:

I want to thank Michael Kovalsky for pointing out that FabricRestClient in Semantic Link supports (since v 0.7.5) Long Running Operation (LRO).

LRO support allows the client to wait for the request to process without being blocked. Without LRO support, you will get a 202 response code saying the request is being processed. You need to submit another request based on the url returned to get the result. With LRO support, FabricRestClient will wait 20s and give you the result back.

Click through to see what you’d need to do to enable it, as well as the benefit you can receive.

Comments closed

Defining the Default Lakehouse for a Fabric Notebook

Published 2024-07-26 by Kevin Feasel

Sandeep Pawar sets up a default lakehouse:

I wrote a blog post a while ago on mounting a lakehouse (or generally speaking a storage location) to all nodes in a Fabric spark notebook. This allows you to use the File API file path from the mounted lakehouse.

Mounting a lakehouse using mssparkutils.fs.mount() doesn’t define the default lakehouse of a notebook. To do so, you can use the configure magic as below:

Read on for that command, as well as some notes around using it.

Comments closed

Calculating the Size of Dataflow Gen2 Staging Lakehouses

Published 2024-07-23 by Kevin Feasel

Sandeep Pawar busts out the calculator:

My friend Alex Powers (PM, Fabric CAT) wrote a blog post about cleaning the staging lakehouses generated by Dataflow Gen2. Before reading this blog, go ahead and read his blog first on the mechanics of it and the whys. Note that these are system generated lakehouses so at some time in the future, they will be automatically purged but until then the users will be paying the storage cost of these lakehouses. If you want to read more about how dataflow gen2 works and whether you should stage or not , read this and this blog.

Read on for a Python script using the SemPy library.

Comments closed

A Glossary of Microsoft Fabric Terms

Published 2024-07-23 by Kevin Feasel

Reza Rad talks terminology:

There are a lot of similar terminologies and words when working with Fabric. We have Data Lake, Delta Lake, OneLake, and Lakehouse, and the list continues. I found it confusing for some to understand the differences between them. Although explaining these terms and their differences requires many different blog articles, having one place for a quick definition for each can be helpful. Hence, this Glossary.

I will keep this updated as we have added new features, tools, and workloads, and I welcome all your input regarding what keywords, terms and terminologies you would like to see added here.

Click through for the video, as well as a list of terms and brief definitions.

Comments closed

Testing GitHub Support in Microsoft Fabric

Published 2024-07-22 by Kevin Feasel

Kevin Chant gives it a try:

By the end of this post, you will see the results of my initial tests. Along the way I share plenty of links and some advice.

If you need help with any jargon used in this post, you can read my other post. Which is a Microsoft Fabric Git integration jargon guide for Fabricators.

One key point I want to highlight is that the GitHub support for Microsoft Fabric Git integration is currently in preview and the contents of this post are subject to change.

It’s been a long time coming, that’s for sure.

Comments closed