A Fine Slice Of SQL Server
Jovan Popovic makes an announcement:
The BULK INSERT statement is generally available in Fabric Data Warehouse. The
BULK INSERTstatement enables you to ingest parquet or csv data into a table from the specified file stored in Azure Data Lake or Azure Blob storage:BULK INSERT table_name FROM file_url_path
Read on for some additional notes around bulk inserting into Fabric Data Warehouses.
Comments closedAndreas Bergstedt shows off a new tool:
In today’s ever-changing analytics landscape it can be difficult to plan out your next project or your enterprise analytics roadmap.
Designed to optimize data infrastructure planning, the Microsoft Fabric SKU Estimator helps customers and partners to accurately estimate capacity requirements and select the most suitable SKU for their workloads, protecting users from under-provisioning and overcommitment.
Click through for a few scenarios of translating your existing warehousing and analytical systems into expected Microsoft Fabric needs.
Comments closedJonathan Garriss has an announcement:
We’re excited to unveil the Microsoft Fabric SKU estimator, now available in preview—an enhanced version of the previously introduced Microsoft Fabric Capacity Calculator. This advanced tool has been refined based on extensive user feedback to provide tailored capacity estimations for businesses.
Designed to optimize data infrastructure planning, the Microsoft Fabric SKU Estimator helps customers and partners accurately assess capacity requirements and select the most suitable SKU for their workloads, protecting users from under-provisioning and overcommitment.
And, in classic Microsoft Fabric fashion, it’s in preview.
Coming up with some fairly low estimates for a lot of things, it bounced me between an F32 and an F64, which seemed about right.
Comments closedYael Biss has an announcement:
Microsoft Purview’s Data Loss Prevention (DLP) policies for Fabric now supports Fabric KQL and Mirrored DBs!
Purview DLP policies help organizations to improve their data security posture and comply with governmental and industry regulations. Security teams use DLP policies to automatically detect upload of sensitive information to Microsoft 365 applications like SharePoint and Exchange, and to Fabric’s semantic models and lakehouses.
In today’s fast-paced data-driven world, enterprises are building more sophisticated data platforms to gain insights and drive innovation. Microsoft Fabric Lakehouses combine the scale of a data lake with the management finesse of a data warehouse – delivering unified analytics in an ever-evolving business landscape. But with great data comes great responsibility. Protecting sensitive information and ensuring regulatory compliance is paramount. That’s where Data Loss Prevention (DLP) policies with restricted access come into play.
Click through to see what this preview currently offers.
Comments closedKevin Chant doesn’t limit us to Azure DevOps:
In this post I want to show how you can operationalize fabric-cicd to work with Microsoft Fabric and GitHub Actions. Since I got asked if this post was available whilst I was helping at the ask the experts panel during the Microsoft Fabric Community Conference.
Just so that everybody is aware, fabric-cicd is a Python library that allows you to perform CI/CD of various Microsoft Fabric items into Microsoft Fabric workspaces. At this moment in time there is a limited number of supported item types. However, that list is increasing.
Click through for a high-level diagram and the process, including the code Kevin used in the GitHub Actions workflow.
Comments closedBoniface Muchendu builds out some capacity:
To begin, we need to head over to the Azure portal. You might wonder why we are starting here. Well, Microsoft Fabric is now an Azure resource, which means all initial setups must be done in the Azure environment.
Click through for step-by-step instructions. Microsoft has also been really good about letting people create (and re-create and re-create) trial capacities, so if you’re just futzing about with the product to get an idea of what it can do, see if you can use that rather than shelling out the cash.
Comments closedNikola Ilic doesn’t accept the default:
In this article, I’ll walk you through how to go from out-of-the-box default Spark configurations to a fine-tuned setup that suits your specific workloads and requirements, as well as getting you ready for the DP-700 exam.
Spark is an extremely powerful engine, but like any powerful tool, it runs best when you tune it. So, don’t always settle for default. Get dynamic—and get Spark working the way you need it to.
Click through for the explanation of functionality.
Comments closedSunitha Muthukrishna announces a new trio of VS Code extensions:
Microsoft Fabric is changing how we handle data engineering and data science. To make things easier, Microsoft added some cool extensions for Visual Studio Code (VS Code) that help you manage Fabric artifacts and build analytical applications.
By adding these Microsoft Fabric extensions to VS Code, developers can quickly create Fabric solutions and manage their data setups right from their coding environments. Here, we’ll look at these extensions and show why they’re useful.
Click through for notes on the three extensions that are available. Note that two of them are still in preview.
Comments closedDennes Torres takes a peek at some views:
You may have already discovered the 4 special views the lakehouse has in the queryinsights schema to track query performance. I made a video about the lakehouse special tables, but since then, they evolved a lot:
- queryinsights.exec_requests_history
- queryinsights.exec_sessions_history
- queryinsights.frequently_run_queries
- queryinsights.long_running_queries
Let’s discover what these tables have to offer for us to analyze the lakehouse performance.
Click through to see what each one of these holds.
Comments closedSomeleze Diko shows off a powerful feature in Microsoft Fabric:
Driving actions from real-time organizational data is important for making informed data-driven decisions and improving overall efficiency. By leveraging data effectively, organizations can gain insights into customer behaviour, operational performance, and market trends, enabling them to respond promptly to emerging issues and opportunities.
Setting alerts on KQL queries can significantly enhance this proactive approach, especially in scenarios such as customer support. For instance, by monitoring key metrics like response times, ticket volumes, and satisfaction scores, support teams can identify patterns and anomalies that may indicate underlying problems.
This helps drive home an important mental shift around “real-time intelligence.” Ignoring my standard disdain for misuse of the term “real-time,” most people will ignore the feature because of a perfectly reasonable belief: my data doesn’t come in that frequently, so I don’t really need to process it in near-real-time. But the real-time intelligence functionality isn’t necessarily just about loading in your data and making it available to users faster. Instead, think of it as acting immediately when your data does change, especially if you have multiple sources of data loading at different times during the day.
Comments closed