A Fine Slice Of SQL Server
Sandeep Pawar builds out some data:
If you want to learn or demo Real Time Analytics in Microsoft Fabric, you will need a streaming data source. You can use the built-in samples to get started. But there are several data generators which you can use to create custom streaming sample datasets, Azure Stream Analytics data generator being one of them. You can see them here. In this blog, I will show how to set one up to use with Fabric Eventstream.
Read on for a step-by-step guide.
Comments closedAndy Leonard asks, what’s in a name?
I admit, this functionality may not be as useful for you as it is to me. Why do I like it so much? Well, I tweeted about it:
Read on for Andy’s reasoning, as well as one way that you can change a Microsoft Fabric workspace name.
If only we could change the name of Microsoft Fabric itself…
Comments closedGerhard Brueckl models some data:
Defining data types and knowing the schema of your data has always been a crucial factor for performant data platforms, especially when it comes to string datatypes which can potentially consume a lot of space and memory. For Lakehouses in general (not only Fabric Lakehouses), there is usually only one data type for text data which is a generic STRING of an arbitrary length. In terms of Apache Spark, this is StringType(). While this applies to Spark dataframes, this is not entirely true for Spark tables – here is what the docs say:
Read through for more information on that, as well as how to define a table in a Microsoft Fabric lakehouse using VARCHAR(). The display is a little weird, but Greg Low explains why in the comments.
Comments closedSandeep Pawar kicks off multiple notebooks at once:
The
notebookclass in mssparkutils has two methods to run notebooks –runandrunMultiple.runallows you to trigger a notebook run for one single notebook. Mim wrote a nice blog to show how to use it and its usefulness.
runMultiple, on the other hand, allows you to create a Direct Acyclic Graph (DAG) of notebooks to execute notebooks in parallel and in specified order, similar to a pipeline run except in a notebook.
Read on to learn more about the advantages of this latter approach as well as how you can do it.
Comments closedSoheil Bakhshi digs into the fine print:
If you are evaluating Microsoft Fabric and do not currently own a Premium Capacity, chances are you’re using Microsoft Fabric Trial Capacities. All Power BI users within an organisation or specific security groups given the rights can opt into Fabric Trial Capacities. Therefore, you may already have several Trial Fabric Capacities in your tenant. Your Fabric Administrators can specifically control who can opt into the Fabric Trial capacities within the Fabric Admin Portal, on the Help and support settings section, and enabling the Users can try Microsoft Fabric paid features setting as shown in the following image:
Read on for a lot more detail, including several common issues you might find along the way.
Comments closedMichael Olschimke and Dmytro Polishchuk continue a series:
The last article in this blog series discussed the basic entity types in Data Vault 2.0: hubs, links and satellites. While it would be theoretically possible to limit a model to just these three basic entity types, the resulting Data Vault model would be inefficient: it would most likely consume too much storage, be less efficient due to the many joins, and require a number of grain shifts during information delivery. This is due to certain characteristics in the data that require special treatment.
For these characteristics, Data Vault 2.0 provides special entity types that deal with the specialities. This article focuses on two of them: the non-historized link, which is used to capture transactions and events, and the multi-active satellite, which is used to model multiple active descriptions for the same parent hub or link in the same load.
Read on for an example of how to implement this in a Microsoft Fabric warehouse.
Comments closedGilbert Quevauvilliers takes us through a comparison:
In this blog post I am going to compare Dataflow Gen2 vs Notebook in terms of how much it costs for the workload. I will also compare usability as currently the dataflow gen2 has got a lot of built in features which makes it easier to use.
The goal of this blog post is to understand which in my opinion is cheaper and easier to use, which will then be the focus for future blog posts with regards to what I’ve learned along the way, which will hopefully assist you too.
To compare between the two workloads, I am going to be using the same source file as well as do the same transformations which will result in the same result.
Read on for a surprising difference in cost.
Comments closedGinger Grant takes us through the Data Activator:
With the GA release of Fabric in November, 2023, I am dedicating several posts to new features which you will not find in Power BI or Azure Synapse, and the latest one I want to talk about is Data Activator. Data Activator is an interesting tool to include inside of Fabric because it is not reporting or ETL, rather it is a way to manage actions when the data hits defined targets. It is a management system for data stored in Fabric or streamed in Azure using IOT or Event Hubs. You can use Data Activator to monitor the current state or to define actions to occur when certain conditions occur in the data. Data Activator is still in preview, but you can evaluate it now.
Read on to see how to enable it and what you can currently do with it.
Comments closedGinger Grant tries out a feature in Microsoft Fabric:
The second element in my series on new Fabric Features is Data Wrangler. Data Wrangler is an entirely new feature found inside of the Data Engineering and Machine Learning Experience of Fabric. It was created to help analyze data in a lakehouse using Spark and generated code. You may find that there’s a lot of data in the data lake that you need to evaluate to determine how you might incorporate the data into a data model. It’s important to examine the data to evaluate what the data contains. Is there anything missing? Incorrectly data typed? Bad Data? There is an easy method to discover what is missing with your data which uses some techniques commonly used by data scientists. Data Wrangler is used inside of notebooks in the Data Engineering or Machine Learning Environments, as the functionality does not exist within the Power BI experience.
Click through to see how it works. I liken it to Power Query for people who don’t like Python.
Comments closedSandeep Pawar has a one-liner for us:
What the title says, thanks to the latest version (v0.5) of Semantic Link
Click through for a quick script that uses a Python list concatenation to do the work. Sandeep also has a simple example and a warning for you, so check that out.
Comments closed