Press "Enter" to skip to content

Category: Microsoft Fabric

Microsoft Fabric SQL Endpoints and REST API

Published 2023-12-12 by Kevin Feasel

Tomaz Kastrun continues a series on Microsoft Fabric. Day 6 covers the SQL Analytics endpoint:

SQL Analytics endpoint in lakehouse is a SQL-based experience for lakehouse delta tables. By using standard T-SQL language, you can write queries to analyze data in delta tables, create functions, procedures, views and even apply security over the objects. There are some of the functionalities missing from your standard T-SQL language, but the experience is the same.

Besides the SQL experience, you can always use the corresponding items in the workspace view of Lakehouse explorer, use SQL in notebooks, or simply use SQL analytics endpoint

Day 7 looks at what subset of T-SQL syntax you can use against SQL Analytics endpoints:

You get the gist, and there are some other limitations; computed columns, indexed views, any kind of indexes, partitioned tables, triggers, user-defined types, sparse columns, surrogate keys, temporary tables and many more. Essentially, all the commands that are not supported in distributed processing mode.

The most biggest annoyance (!) is case sensitivity! Ughh.. This proves that the SQL operates like API on top of delta tables, which is translated either into PySpark commands or not directly to Spark since Spark is not case-sensitive. So, the first one will work and the second statement will be gracefully terminated.

Day 8 covers the Lakehouse REST API:

Now that we explored the lakehouse through the interface and workspaces, let’s check today, how can we use REST API. Microsoft Fabric Rest API defines a unified endpoint for operations.

Comments closed

Security in Microsoft Fabric

Published 2023-12-12 by Kevin Feasel

Alex Lisboa-Wright talks security:

In Fabric, the basic logical structure for data services is the Workspace, in which users can create Items, which are the various resources that perform all the data operations available in Fabric, such as Lakehouses, pipelines, machine learning models and so on. Each workspace is a self-contained data storage and development environment, whose user access is controlled by both workspace admins and member users. User access controls include options to manage users’ workspace roles, which determine the permissions assigned to each user. Security permissions can be managed on the workspace and item levels in the Fabric UI. MEID authentication can also be employed within Fabric, as connecting Fabric items to other Azure resources requires MEID. MEID’s Conditional Access feature can also be configured for use in Fabric (see this documentation for best practice for Fabric resources linking to other Azure services).

Read on to learn more. Fabric is a broad set of tools and technologies, so security is both important and definitely non-trivial, even when you consider that it is a software-as-a-service offering and therefore doesn’t have much going on with user-facing networking or infrastructure security.

Comments closed

An Overview of Lakehouses in Microsoft Fabric

Published 2023-12-11 by Kevin Feasel

Kevin Chant invites you to a swank lakhouse:

By the end of this post, you will have a good overview of Microsoft Fabric Data Lakehouses, including CI/CD options. In addition, where your SQL Server background can prove to be useful and where some Power BI knowledge can come in handy.

Plus, I share plenty of links in this post. For instance, there are a couple of useful resources to help you get started towards the bottom of this post.

Click through for the article.

Comments closed

Refreshing Individual Tables and Partitions using Semantic Link

Published 2023-12-08 by Kevin Feasel

Sandeep Pawar doesn’t have time to refresh everything:

The latest version of Semantic Link (0.4.0) has many methods that provide a convenient abstraction for calling Fabric/Power BI REST APIs. You can see them here. In this blog, I will show how to use the .refresh_dataset() which uses the Enhanced Refresh API to refresh Power BI semantic models, tables and partitions.

Read on for two ways to do it.

Comments closed

All about Lakehouses in Microsoft Fabric

Published 2023-12-07 by Kevin Feasel

Tomaz Kastrun gives us the skinny with multiple posts in his Advent of Microsoft Fabric. Day 3 introduces the lakehouse:

Lakehouse is cost-effective and optimised storage, supporting all types of data and file formats, structured and unstructured data, and helps you govern the data, giving you better data governance. With optimised and concurrent reads and writes, it gives outstanding performance by also reducing data movement and minimising redundant copy operations. Furthermore, it gives you a user-friendly multitasking experience in UI with retaining your context, not losing your running operations and working on multiple things, without accidentally stopping others.

Day 4 covers Delta format:

Yesterday we looked into lakehouse and learned that Delta tables are the storing format. So, let’s explore what and how we can go around understanding and working with delta tables. But first we must understand delta lake.

Day 5 covers data ingest:

We have learned about delta lake and delta tables. But since we have uploaded the file directly, let’s explore, how we can also get the data into lakehouse.

Click through for all three posts.

Comments closed

Visualizing a Power BI Refresh with the semantic-link Library

Published 2023-12-06 by Kevin Feasel

Phil Seamark builds a notebook:

A few blogs back I shared a technique using Power BI Profiler (or VS Code) to run and capture a trace over a refresh of a Power BI semantic model (the object formally known as a dataset).

I’ve since received a lot of positive feedback from people saying how useful it was to visualize each internal step within a problematic Power BI refresh. Naturally, in the age of Fabric, I’m keen to share how the same approach works using a Microsoft Fabric Notebook.

Click through to see how you can do it.

Comments closed

Enabling Microsoft Fabric

Published 2023-12-05 by Kevin Feasel

Tomaz Kastrun continues a series on Microsoft Fabric:

If you have used Power BI services in the past, you will be on board immediately. The outlook is the as it is with the Power BI. You will only need additional credentials to access the services. In general, you will need Azure subscription, Power BI service already enabled, and the ability for your organization to enable Fabric with Admin roles

Click through to see how to enable Microsoft Fabric in your environment.

Comments closed

Generating Reports in Azure ML with Copilot

Published 2023-12-04 by Kevin Feasel

Soheil Bakhshi automates report creation:

In Nov 2023, Microsoft announced Microsoft Fabric’s general availability and Public Preview of Copilot in Microsoft Fabric. In a previous post, I explained what Copilot means to Power BI developers, which is valid for other Fabric developers such as data engineers and data scientists as Copilot for Fabric helps with those experiences as well. But the main focus of this blog post is to discuss the requirements, how to enable Copilot, and how to use it from a Power BI development point of view. So, this blog will not discuss other aspects of Copilot in Microsoft Fabric. With that, let’s begin.

I haven’t been particularly impressed with the reports it generates, but I suppose this is like the proverbial bear riding a unicycle: it’s not a question of how well it does the task that makes it interesting, but rather that it does it at all.

Comments closed

What Is Microsoft Fabric?

Published 2023-12-04 by Kevin Feasel

Tomaz Kastrun starts a new series:

Microsoft Fabric is a next-gen platform, that brings all-in-one data and analytics solution for end users, small, medium and large enterprises. Services offer the complete data cycle movement (data ingestion, data engineering, data integration, data storing with warehouse using one lake), delivering data insights and building predictive models.

Read on for the overview and stay tuned for plenty more where that came from.

Comments closed