A Fine Slice Of SQL Server
In this article, you will learn how to train a Scikit-learn classification model, serve it with FastAPI, and deploy it to FastAPI Cloud.
Topics we will cover include:
- How to structure a simple project and train a Scikit-learn model for inference.
- How to build and test a FastAPI inference API locally.
- How to deploy the API to FastAPI Cloud and prepare it for more production-ready usage.
Click through for the process.
Leave a CommentIf you’ve been using Database Projects for simply maintaining copies of your database objects – like I used to – then you’ve been missing out on the power of its deployments. Initially I was sceptical about how it could effectively perform upgrades but after seeing them land in SSMS last month I wanted to revisit them as a means of actual deployment.
My scepticism was completely misplaced, so if you haven’t touched Database Projects before – or had similar concerns as me – I want to demonstrate 3 features which make them not just good, but amazing for deployments.
Click through for those features. I will say that for straightforward databases, the database project deployment process is pretty good. Where it falls apart is when you have a large number of cross-database dependencies, especially if there are mutual cross-database dependencies: DB1.dbo.sp1 needs DB2.dbo.Table2, and DB2.dbo.view2 references DB1.dbo.Table1. In that case, the workaround is so annoying and essentially comes down to “have three separate database projects, one for DB1, one for DB2, and one for a scalled-down version of DB1/DB2 without the dependencies, and then use that to inject into the other DB.” Which does kind of work, yeah, but now you’re maintaining even more. And once you get to dozens of dependencies and lots of cross-database queries? Yeah, forget about it.
Leave a CommentThis post covers using fab deploy in Azure DevOps for Microsoft Fabric deployments based on YAML pipelines. In addition, this post shows how you can perform initial tests locally and introduces some AI concepts. Plus, this post shares plenty of links and advice.
You can find an example to accompany this post in the ‘create-genworkspace-fabric-cli.yml‘ file my ADO-fabric-cicd-sample Git repository. I also added some AI elements within this Git repository as well. Including the Fabric CLI skills that were announced during FabCon.
Click through to learn more about fab deploy and how the process works.
Leave a CommentGilbert Quevauvilliers works with the on-again, off-again CI/CD solution Microsoft has to offer:
In this blog post is a way set up version control for Power BI semantic models (and reports) using the PBIP (Power BI Project) format, Azure DevOps (Azure Repos), and VS Code.
This approach treats your semantic model as readable text files (JSON/TMDL), enabling proper Git diffing, branching, merging, and collaboration—something binary .pbix files don’t support well.
Click through for the process.
Comments closedKevin Chant announces a new Azure DevOps extension:
This post covers how you can simplify Microsoft Fabric deployments with “Deploy Microsoft Fabric items with fabric-cicd”. Which is an Azure DevOps extension that I recently published.
To manage expectations, this post shows how to start working with the extension and its associated task within the GUI-based classic release pipelines in Azure DevOps. Like in the below screenshot.
Read on to see how the extension works.
Comments closedKoen Verbeeck glues together several technologies:
Quite the title, so let me set the stage first. You have an Azure Data Factory instance (or Azure Synapse Pipelines) and you have a couple of linked services that point to Fabric artifacts such as a lakehouse or a warehouse. You want to deploy your ADF instance with an Azure Devops build/release pipeline to another environment (e.g. acceptance or production) and this means the linked services need to change as well because in those environments the lakehouse or warehouse are in a different workspace (and also have different object Ids).
When you want to deploy ADF, you typically use the ARM template that ADF automatically creates when you publish (when your instance is linked with a git repo). More information about this setup can be found in the documentation. To parameterize certain properties of a linked service, you can use custom parameterization of the ARM template. Anyway, long story short, I tried to parameterize the properties of the Fabric linked service.
Read on to see how that went, as well as what you need to do to solve this issue.
Comments closedHamish Watson lays out what DevOps means in the context of Microsoft Fabric:
Microsoft Fabric (not to be confused with the more general term “fabric” in DevOps) is an integrated data and analytics platform designed for modern data-driven workloads, such as data engineering, business intelligence, and machine learning. With the introduction of Git integration in Microsoft Fabric, DevOps practices are becoming more accessible in the platform, allowing teams to implement collaborative, automated workflows that are common in DevOps environments.
Read on for some of the high-level concepts of what we do with DevOps and how they apply directly to Microsoft Fabric workspaces.
Comments closedRobin Moffatt shares a tutorial on dbt:
In 2022 I made a couple of attempts to learn dbt, but it never really ‘clicked’.
I’m rather delighted to say that as of today, dbt has definitely ‘clicked’. How do I know? Because not only can I explain what I’ve built, but I’ve even had the 💡 lightbulb-above-the-head moment seeing it in action and how elegant the code used to build pipelines with dbt can be.
In this blog post I’m going to show off what I built with dbt, contrasting it to my previous hand-built method.
I also had heard of dbt but haven’t really spent the time to learn it because I’m not really a data engineer. But this tutorial has me interested in diving in further.
Comments closedYaron Pri Gal announces support for a library:
Today, we’re announcing that fabric‑cicd—the open‑source Python deployment library for Microsoft Fabric—is now an officially supported, Microsoft‑backed tool for CI/CD automation across Fabric workspaces.
Over the past year, fabric‑cicd has rapidly evolved through collaboration with engineering, CAT, MVPs, enterprise customers, and the community. Growing usage, strong sentiment across internal and external channels, and adoption by organizations building enterprise‑grade deployment pipelines helped solidify its value within the Fabric ecosystem.
Read on to learn what this means.
Comments closedOlivier Van Steenlandt runs into an issue:
Over the past few weeks, I have been doing some experimenting with Azure SQL mirroring to Microsoft Fabric. In the process, I ran into a couple of issues and challenges. In this data recipe, I will be going through one of my challenges when I got Azure SQL mirroring to Microsoft Fabric setup and running.
At first, everything seemed to be working as expected, and the integration felt very smooth. At that point, I continued to develop my test database in Azure SQL to learn a bit more about mirroring. I made a couple of minor changes to my test database and tried to publish them from my SQL Database Project.
Read on for the issue, as well as the solution—that is, assuming you don’t actually want to change any of the things you’re actively moving over into Fabric.
Comments closed